The Story of an AI-Creative Partner in Watermarking Architectures: An Empirical Approach to Explain the Story of a Human-Interacting Machine
People working in the artificial intelligence space are hesitant about watermarking, as flaws in it haven’t discouraged tech giants from offering it. When it comes to real-world applications, watermarking is not a good solution since they can be easily faked, removed, or ignored.
So, it was really amazing the extent to which having an AI creative partner helped unblock me. It was still, though. It was all my trying to figure out how the plot of this book ought to work. I do not think it would be interesting for a reader to consume a book with no human touch, but was 100 percent created by an artificial intelligence. I am not sure what that is doing.
Why are People Making Art? Why do people make art? And what do we want to see in the GPU, in the cloud, at the end of the day?
We’ve arrived now at the nature of art, so I’m going to make a hard shift to GPUs. We can go everywhere with Kevin. I just want to make sure we hit it all.
Why are people making art? The opportunity to ask a serious question was given by the Artificial Intelligence moment. Because the internet has basically been like, “To make money.” And I think there’s a divergence there, as our distribution channels get flooded. I do not think we will hit the answer in the next 10 minutes.
So, the last time you and I spoke, you said something to me that I have been thinking about ever since. Every dollar that goes into the hardware is controlled by the man here at Microsoft.
It’s easier now than when we talked last time. We were at a point where I thought the demand was there. Because demand was far exceeding supply, due to a bunch of AI technology ripping onto the scene. That is resolving. It’s still tight, but it’s getting better every week, and we’ve got more good news ahead of us than bad on that front, which is great. It makes my job a little easier to adjudicate gnarly conflicts.
This week there was some reporting. You actually mentioned it before,
in The information
, that Microsoft is heavily invested in smaller models that require less compute. Are you reducing the cost of compute over time?
I think we are. And the thing that I will say here, which we were chatting about backstage, is when you bill one of these AI applications, you end up using a full portfolio model. So, you definitely want to have access to the big models, but for a whole bunch of reasons. If you can reduce some of the work that the application is meant to do, you might want to do it.
And some of the motivations could be cost. Some of that could be a problem. It may be that you don’t want sensitive information to go to the cloud because you want to run part of the application locally. There are a lot of reasons to want flexibility in the way that you architect things.
The people at OpenAI, with some help from the Microsoft team, are working hard to improve the big models as well. It is not an either-or. You want both, and you want both to be getting cheaper and faster and more performance and higher quality over time.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
OpenAI and Google: AI Pricing of Autonomous Machines and Machines for the End of the Industrial Era? Kevin Scott on Bing’s quest to beat Google and the future of AI art
I’m looking at Copilot in Office 365. It is $30 for a seat. That is an insane price. Some people will think that it is valuable but not a large market for an AI pricing scheme. Is it possible to bring that down?
I believe that we can bring the underlying cost of the artificial intelligence down quite a bit. One of the interesting things that OpenAI did this spring is they reduced the cost by a factor of 10 to developers for access to the GPT-3.5 API. That passed along a bunch of performance optimizations. The chip is getting better price performance over generation. And the software techniques that we’re using to optimize the models are bringing tons of performance without compromise to quality down. There are other techniques of how to compose the application of small and big models, as well. So yeah, definitely, the cost goes down. What value you’re creating for people is what the price is. The price is set by the market. And if the market tells us that the price for these things is too high, then the price goes down.
Yeah, we’re getting really good signal about price right now. The thing that you just said is important, I think. It is very early days right now for the commercialization of generative AI. You have a lot of things to figure out in the same way. One of them is how do you price them, and what is the market actually for these things? There is no reason to overprice things. Everybody getting value from them is what you want. So we’ll figure that out, I think, over time.
I think about compute when I think about the story about the Nvidia chips. It is access to H 100s. It’s building capacity there. 80 percent of the market share is theirs. How much do they represent for you?
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
How do you look at first-party semiconductors? Is it CUDA versus GPUs, what do you think about it?
They are one of the most important partners. And we work with them on a daily basis, on a whole bunch of stuff, and I think the relationship is very good.
They make their own chips, that is what I look at. I spoke to the CEO of the company a couple of weeks ago. He didn’t sound thrilled that he had this existential dependency on Nvidia. They don’t want to work with other people’s systems. Are you thinking about custom chips? Are you thinking about changing that supply chain for yourself?
Going back to the previous conversation, if you want to make sure that you’re able to price things competitively, and you want to make sure that the costs of these products that you’re building are as low as possible, competition is certainly a very good thing. I know that Lisa Su works for Advanced Micro Devices at the conference. We’re doing a bunch of interesting work with Lisa, and I think they’re making increasingly compelling GPU offerings that I think are going to become more and more important in the marketplace in the coming years. I think there’s been a bunch of leaks about first-party silicon that Microsoft is building. We’ve been building silicon for a really long time now. So—
I’m not confirming anything. But I will say that we’ve got a pretty substantial silicon investment that we’ve had for years. And the thing that we will do is we’ll make sure that we’re making the best choices for how we build these systems, using whatever options we have available. And the best option that’s been available over the past handful of years has been Nvidia. They have been really—
Is it due to the processing power in the chip, or is it because of the platform? Because what I’ve heard from folks, what I heard from Lisa yesterday, is that actually, what we need to do is optimize one level higher. We need to optimize at the level of PyTorch or training or inference. The thing is not CUDA, it’s perceived mode. Do you think that is right? Do you depend on the chip? Are you dependent on their software infrastructure? Are you at a higher level than that?
Well, I think the industry at large benefits a lot from CUDA, which they’ve been investing in for a while. It’s important for your business to have a PyTorch-CUDA combo if you want to perform tune all of your models. We don’t have a lot of models in our arsenal.
So we have a whole bunch of other tools like Triton, which is an open-source tool that OpenAI developed, and a bunch of other things that help you basically do exactly what you said, which is up-level the abstraction so that you can be developing high-performance kernels for your both inference and training workloads, where it’s easier to choose what piece of hardware you’re using. The thing to remember is even if it’s just Nvidia, you have multiple different hardware SKUs that you’re deploying in production at any point in time, and you want to make it easy to even optimize across all of those things.
I asked how easy it would be for Microsoft to switch to AMD. And she told me, “You should ask Kevin that question.” So here you are. Would it be difficult to switch to the company of your choice? Are you working with them on anything? Can it be done in the future?
This hardware is not trivial to deal with. It’s all investments. If that is the way that you’re building your application, you don’t need to care. A bunch of people are not building on top of the protocols and there is a need for their care. And then, that’s a choice for all of them individually about how difficult they think it might be. But for us, it’s a big complicated software stack, and the only part of that that the customer sees is that API interface.
The open source theme was brought up by a bunch of people yesterday at the conference. You obviously have a huge investment in your models. OpenAI has GPT. There is a lot of action going on. There are some open-source models that are really exciting. You were talking about running models locally on people’s laptops. Are these real obstacles for these big models right now? Or is open source going to actually just come and disrupt it over time?
I don’t know if it’s important to think about the models like moats. So there are some things that we’ve done, and a path forward for the power of these models as platforms, that are just super capital intensive. Even if you have a lot of breakthrough on software, I don’t think it becomes less capital intensive. So, whether it’s Microsoft or someone else, the thing that will have to happen with all of that capital intensity… because it’s largely about hardware and not just software, and it’s not just about what you can put on your desktop — is you have to have very large clusters of hardware to train these models. It’s hard to get scale by just sort of fragmenting a bunch of independent software efforts.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
Some Questions on Cryptographic Watermarking from Artificial Intelligence at the Bottom of an Image or a Video: Kevin Scott on Bing’s quest to beat Google and the future of AI art
I have a few more questions. Please start asking questions if you have any. I would love to hear from you. I want to make sure that we are talking about what is real, and what is not. and I have talked about a lot in the past. There are many ways in which you could mark content as real or generated by Artificial Intelligence. We are going to see some later today from Adobe. Have you made any progress?
I believe we have. One of the things I think we talked about before is for the past handful of years, we’ve been building a set of cryptographic watermarking technologies and trying to work with both content producers and tool makers to see how it is we can get those cryptographic watermarks — they’re manifests that say, “This piece of content was created in this way by this entity” — and have that watermark cryptographically preserved with the content as it gets moved through transcoders and CDNs and as you’re mashing it up a bunch of different ways.
It is more difficult to read text. There are some research projects that can be done if you can add a statistical fingerprint to the way you generate the text. It’s much more difficult to hide a watermark in an image than in a video, where you can just hide the noise in the picture and not have it affect the experience you have. So it’s a tougher problem, for sure.
There’s nothing I want more than someone sending me an email that says it was generated from AI at the bottom. When I think about my inbox, that’s what would fix it.
I know what my opinion of those emails is. I’m going to tell Cortana to delete ‘em right away. Fair warning to all of you. If you write me something, it is gone.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
Kevin Scott on Bing to beat Google and the future of AI art: Expert Contributions to Artificial Intelligence in Consumer-Centered Requirements
Pam Dillon is pregnant. Good morning, Kevin. Pam Dillon of Preferabli. This question is not being generated by ChatGPT. We are talking a lot about assimilating the world’s knowledge. Do you think about how we’re going to start to integrate specialized bodies of knowledge areas where there’s real domain expertise? Say, for example, in medicine or health, demands a sensory consumer?
Kevin Scott is not a person. Yeah, we are thinking a lot about that. And I think there’s some interesting stuff here on the research front that shows that those expert contributions that you can make to the model’s training data, particularly in this step called reinforcement learning from human feedback, can really substantially improve the quality of the model in that domain of expertise. We’ve been thinking in particular a lot about the medical applications.
Peter Lee wrote a book about medicine, GPT-4 and a lot of other good things, and he is one of my direct reports. All of that is exactly what you said. It is how — through reinforcement learning, through very careful prompt engineering, through selection of training data — you can get a model to be very high performing in a particular domain. And I think we’re going to see more and more of that over time, with a whole bunch of different domains. It is very exciting.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
Is the Black Keys Music Used in the Generating of their Output? An Articulation of Robert Kyncl: What Do We Know About Provenance?
Alex: Hi Kevin, my name is Alex. I have a question about provenance. Yesterday, the CEO of Warner Music Group, Robert Kyncl, was talking about his expectation that artists are going to get paid for work that is generated off of their original IP. Today, obviously, provenance is not given by LLMs. My question to you is from a technical standpoint: Let’s say that somebody asks to write a song that’s sort of in the style of Led Zeppelin and Bruno Mars. The Black Keys are a band that sound a lot like Led Zeppelin, so the LLM is using their music. Would there be a way, technically, to be able to say, from a provenance standpoint, that the Black Keys’ music was used in the generating of the output so that artist could get compensated in the future?
KS: Maybe. Although, that particular thing that you just asked, I think, is a controversial thing for human songwriters. It is very easy for a human writer to be influenced in subtle ways when it comes to this. And a lot of pop songs, for instance, have a lot of harmonic similarity with one another.
So, I think you have to think about both sides of things. How do you measure contribution of one thing to another and what is the impact on other things? Which is hard. If we were able to do that part of the analysis, you probably could figure out some technical solutions. It is very easy to make sure that you do not have generations that are parroting. It can either be in whole or in snippets. It is more difficult to figure out how a single piece of data has influenced a particular generation, because it is a large amount of contribution.
I would bet that no one of you could recite the third paragraph of the fifth printing of Moby Dick. And these neural networks work a little bit like that. They’re not storing the content of books or music that people are generating the same way that a search engine is. They are ingesting some of these things. Everyone believes that all of the training that is being done at the moment is covered by fair use and that will be a part of what we decide over the coming years.
Here is a thought exercise. By raise of hands, how many of you have read Moby Dick? So, I’m guessing that all of you who raised your hand probably read Moby Dick many, many years ago — high school, college maybe. You could tell me that Moby Dick is about a whale. There’s a captain. Maybe you remember his name is Ahab. He might have some sort of fixation issue with this object. You could tell me a bunch of things about Moby Dick. Some of you who are literature fans might even be able to recite a passage or two from Moby Dick exactly as they appear in the book.
KS. And that’s the thing that will get sorted out. And I don’t know the answer to that question because it relies on judges and lawmakers, and we will sort of figure this out as a society. The thing the models are attempting to do is not… They’re not some gigantic repository of all of this content. You are trying to build something that you can remember conceptually the things that were presented in the training. We will have to check it out.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
Managing Artificial Intelligence: How to Make Sense of it and Why to Write It? The Case of a Contributor who Writes Nonfiction
As an author, I don’t want someone to be left out. Producing content and being able to earn money writing books is an economic incentive for people. Especially, God forbid, folks who sit down and do the work of writing a really thoughtful, super well-researched piece of nonfiction. Or someone who pours their heart and soul into writing a piece of fiction. They should be paid for it. And this is a new modality of what you’re doing with content. We have some big questions to ask about what is going on, what is the fair way to compensate for what is happening, and so on.
And then, what’s the balance of trade, too? We want to build things that will create amazing new ways for creative people to do what they like, which will make us better people and help us connect with each other.
I think the thing that you want in general is, as a consumer of content, you just don’t want to be reading a bunch of spammy AI-generated garbage. I don’t think anyone wants that. I would argue… This is an interesting thing you and I haven’t chatted about, but I think the purpose of making a piece of content isn’t this flimsy transactional thing that sometimes people think it is. It is trying to put something meaningful out into the world, to communicate something that you are feeling or that you think is important to say and then trying to have some kind of connection with who’s consuming it.
There is a person named KS Yeah. It was more about how the human part of the thing was doing, than it is about the artificial intelligence. There would have been a little better if there had been more artificial intelligence.
I am not blaming anyone. I think the diagnosis of that problem is some of these things on MSN — and I know this is true for other places — gets generated in really complicated ways. It wasn’t the case of: there was, at some point, a Columbia-trained journalist who was sitting down writing this, and all of a sudden, there was now a faulty, defective AI tool that was doing the thing that they used to do. This wasn’t what was happening here.
The author is KS Well, I think you all are going to judge the quality of the content. Is it a good or bad thing if it’s directed at you? Is it true or false? One of the things that these artificial intelligence tools could prove to be useful at is actually helping navigate a world that is going to be filled with a lot of low-quality content, one of the things I will plant with you all. I think having a personal editor-in-chief that you can call on will be useful in helping you sort through this ocean of information and find out what you think are high-quality, reliable sources of information. I think that having all of this stuff out there makes your job more important, especially since you all are in media businesses.
KS: Way more important. A person needs someone that they trust and who has high editorial standards to figure out signal and noise. It is absolutely true.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
AI-generated content in a science fiction book: How do I go about it? A comment on GPT-4 and GPT-6: a helpful tool for writing nonfiction books
Correct. And I actually agree with that. But the point that I was making is the useful thing about the tool is it helped keep me in flow state. So I’ve written a nonfiction book. I’ve never written a fiction book before. So the useful thing for it was not actually producing the content but, when I got stuck, helping me get unstuck, like if I had an ever-present writing partner or an editor who had infinite amounts of time to spend with me. It is like, I don’t know the name of the character. Let me describe what they’re about. Give me a bunch of names.
That is the model today. The writers strike is happening now. They weren’t concerned about the model’s capabilities today. There will be a GPT-5 and a GPT-6, right?
I think you are almost certainly going to want to use some of these AI tools to help produce content. One of the things that I did last fall when we were playing around with this stuff for the first time is: I was like, “Oh, I’ve wanted to write a science fiction book since I was a teenager, and I’ve never been able to just sort of get the activation energy.” And I started to attempt doing that with GPT-4, and it was terrible at using it in the way that you would expect. You can only do that if you have an outline for a science fiction book you want to write. Please write chapter one.
It doesn’t seem interesting to me that an ai is the only part of that interaction. I don’t know why I would want to be consuming a bunch of AI-generated content versus things that you are producing.
Sometimes, AI-generated content is good, and sometimes, it’s not. I don’t think it’s quite as interesting. It is a technical issue, whether or not you are consuming stuff into your training process that is making the model performance worse over time. That’s just a technical thing. I think it’s an entirely solvable problem.
We’ve got an increasingly good set of ways, at least on the model training side, to make sure that you’re not ingesting low-quality content, and you’re sort of recursively getting—
There are a lot of tools that can be used to create artificial intelligence. You can see the distribution platforms being flooded with artificial intelligence content. There is a correlation between a search engine and model collapse, as well as a reduction in quality when a model is flooded with its own artificial intelligence. How do you filter that stuff out?
So, I think that’s one of the opportunities that we can have right now in the conversation about how these AI agents are going to show up in the world. It’s not necessarily preserving exactly what that funnel looks like but being transparent about what the mechanics of it are so that if you’re going to spend a bunch of effort or try to use it as a way to acquire an audience, that you at least understand what’s going on, that it’s not arbitrary and capricious and, one day, something changes that no one told you about You don’t know how to run a business.
I think the compensation structure and how things work are changing very quickly. It feels like things are changing very fast, like how people find an audience for things that they are creating, and how people turn audience engagement into a real business model. It is difficult on the one hand because of some of the funnels. You don’t really know what’s going on in the engine that directs traffic to your site.
It is not the thing that anyone would want. It’s certainly not the thing that I want, individually. There needs to be a healthy economic engine where people are all participating. They’re creating stuff, and they’re getting compensated for what they create.
Source: Microsoft CTO Kevin Scott on Bing’s quest to beat Google and the future of AI art
Search as a Platform: What Do I Want to See in a Market Where I Can Get What I Write About My Phone, or What Would I Wish I Had In My New Phone?
If an artificial intelligence product can just summarize for you what I wrote in a review of the new phone, it makes sense that I would write another review of it if no one was going to see it.
Like you are planning a trip, you are researching on how to get the cables out of the house you are remodeling. That may involve purchasing some things or spending some time reading a pretty long thing because you can’t get the information that you need in just some small transaction that you’re having with an agent. I think it’s unclear the extent to which the dynamic will actually change. Everybody is worried about referrals, how is this going to happen? If the bot answers all your questions, what does that mean for referral traffic?
Yeah. So I think what you want from a search engine and what you’re going to want from an agent is a little more complicated than just asking a question and getting an answer. Sometimes asking questions is part of a task, but it is not the end of it. Sometimes, it’s in the middle.
I think the conventional wisdom is that [in] an AI-powered search experience, you ask the computer a question, it just tells you a smart answer, or it goes out and talks to other AI systems that sort of collect an answer for you. That is the future. I think if you just broadly ask people, “What should search do?” “You ask a question, you get an answer.” That really changes the idea of how the web works. The fundamental incentive structure on the web is appearing in search results. Is that something that you have thought about with Bing?
Yeah, broadly. But I do think we have to be asking ourselves all the time about what’s fair and how can everyone participate. Because that’s the goal at the end of the day. We’re all building big platforms, whether it’s search as a platform, or cloud platforms, we’re building them. I think everybody is very reasonable in wanting to make sure that they can use these platforms in a fair way to do awesome work.
And I think the only thing that anybody can ask for is that you do high-quality product work, and you want marketplaces to be fair so you can compete. I think it is true for large companies and individuals who are trying to break through. Just whatever it is that is that notion of fairness is what everybody’s asking for. It is difficult to go sort out. I will not comment on what’s going on on the East Coast right now.
The context of this question is, as we sit here on the West Coast having this conversation, on the East Coast, Google is in the middle of an antitrust trial about how it might’ve unfairly created a monopoly in search. And a huge theme in that trial is, “Well, hey, Microsoft exists. They could compete if they wanted to. We are so good at this that they can’t. Do you think Bing actually creates an edge in that race right now?
I could talk about literally anything with Kevin. He’s a maker. You’re a renaissance… Before we took the stage, we were talking about how to crimp ethernet cables. We have to discuss artificial intelligence. So I want to just ask from the beginning: Microsoft kicked off a huge moment in the AI rush with the announcement of Bing, the integration of OpenAI into the products. There’s obviously Copilots. How is that going? Has the integration of AI into Bing led to a market share gain, led to increased usage?
I also asked Kevin some pretty philosophical questions about AI: why would you write a song or a book when AI is out there making custom content for other people? Well, it’s because Kevin thinks the AI is still “terrible” at it for now, as Kevin found out firsthand. But he also thinks that creating is just what people do, and AI will help more people become more creative. I like talking to Kevin and this conversation got deeper.
I co-hosted the Code Conference last week, and today’s episode is one of my favorite conversations from the show: Microsoft CTO and EVP of AI Kevin Scott. If you caught Kevin on Decoder a few months ago, you know that he and I love talking about technology together. I really appreciate that he thinks about the relationship between technology and culture as much as we do at The Verge, and it was great to add the energy from the live Code audience to that dynamic.
Kevin and I talked about how Bing is doing now that the initial hype is over, I was curious about if it was stealing users from Google.
Kevin also controls the entire GPU budget at Microsoft, and access to GPUs is a hot topic across the AI world right now — especially access to Nvidia’s H100 GPU, which is what so many of the best AI models run on. Microsoft itself runs on H100s, but Kevin is keenly aware of that dependency, and while he wouldn’t confirm any rumors about Microsoft developing its own AI chips right now, he did say a switch from Nvidia to AMD or other chip vendors should be seamless for Microsoft’s customers if the company ever does make that leap.