On April 17th, the 12th Old Friends Reunion, themed “Singularity: AI x Crypto Convergence,” was successfully held by IOSG Ventures. We invited outstanding representatives from the industry who are emerging as leaders. The purpose of this gathering was to explore the integration of artificial intelligence and cryptocurrency and its impact on the future. During such events, participants have the opportunity to share their insights, experiences, and ideas, promoting collaboration and innovation within the industry.
Next, let’s talk about one of the keynote speeches at this event. Illia Polosukhin, co-founder of NEAR Protocol from IOSG Ventures’ portfolio, delivered a speech titled “Why AI Needs to be Open – 为何AI需要Web3.”
Let’s explore why artificial intelligence needs to be open. My background is in machine learning, and I have been involved in various machine learning projects for about ten years in my career. However, before stepping into the field of crypto, natural language understanding, and founding NEAR, I worked at Google. We developed a framework called Transformer, which powers most modern artificial intelligence. After leaving Google, I started a machine learning company to teach machines programming and change the way we interact with computers. However, we didn’t do it in 2017 or 2018 because it was too early, and we didn’t have the computational power and data to do so.
At that time, what we did was to attract people from around the world to annotate data for us, mostly students from China, Asia, and Eastern Europe. Many of these people didn’t have bank accounts in their countries. The United States was not willing to transfer money easily, so we started to consider using blockchain as a solution to our problem. We wanted to pay people globally in a programmatic way, making it easier for them no matter where they were located. By the way, the current challenge with crypto is that although NEAR has solved many problems, in general, you need to purchase some crypto first to transact on the blockchain and earn, which is the opposite of what we intended.
Just like companies would say, “Hey, first, you need to buy some shares of the company to use it.” This is one of the many problems NEAR is solving. Now, let’s dive deeper into the field of artificial intelligence. Language models are not something new; they have existed since the 1950s. They are statistical tools widely used in natural language processing. For a long time, starting from 2013, a new innovation began with the revival of deep learning. This innovation allows you to match words, add them to multidimensional vectors, and transform them into mathematical forms. This works well with deep learning models, as they mainly involve matrix multiplication and activation functions.
This enables us to start advanced deep learning and train models to do many interesting things. Looking back now, what we were doing back then were neural networks, which were largely inspired by human models, where we read one word at a time. So, it was very slow, right? If you try to show some content to users on Google.com, nobody would wait for five minutes to get an answer, like reading Wikipedia. Instead, you want the answer immediately. Therefore, Transformer models, which drive ChatGPT, Midjourney, and all recent advances, are based on the same idea, aiming to have parallel processing of data, inference, and immediate answers.
The main innovation here is that each word, each token, and each image block are processed in parallel, leveraging our highly parallel computing capabilities in GPUs and other accelerators. By doing this, we can reason about it at scale. This scalability enables larger training sets to handle automated training. Therefore, after that, we saw Dopamine, which made incredible progress in reasoning and understanding world languages in a short period of time, thanks to the massive amount of text it had.
Now, the direction is to accelerate innovation in artificial intelligence. Previously, it was a tool used by data scientists and machine learning engineers, who would then explain it to their products or discuss data content with decision-makers in some way. Now, we have an AI that directly interacts with people. You may not even know that you are interacting with a model because it is hidden behind the product. So, we have gone through this transition from understanding how AI works to understanding and being able to use it.
When I say we use GPUs to train models, it’s not the kind of GPU you use to play video games on your desktop. Each machine typically has eight GPUs, which are connected to each other through a motherboard and stacked into racks, with about 16 machines per rack. Now, all these racks are also connected to each other through dedicated network cables to ensure information can be transmitted between GPUs at lightning speed. So, it doesn’t fit on a CPU. In fact, you don’t process it on a CPU at all. All the computation happens on the GPU. So, it’s a supercomputing setup. Again, emphasizing that it’s not the traditional “Hey, this is a GPU thing.” So, a model like GPU4, trained for about three months using 10,000 H100s, costs around $64 million. This gives you an idea of the scale of current costs and the expenditure involved in training some modern models.
Importantly, when I say the systems are interconnected, the current connection speed of H100, the previous generation product, is 900GB per second, while the connection speed between the CPU and RAM within a computer is 200GB per second, both of which are local to the computer. So, the speed of sending data from one GPU to another within the same data center is faster than your computer. Your computer can basically communicate within its own box. The connection speed of the next-generation product is about 1.8TB per second. From the perspective of developers, this is not an individual computing unit. These are supercomputers with massive memory and computational capabilities, providing you with tremendous scale of computation.
Now, this leads us to the problem we face, which is that these big companies have the resources and capabilities to build these models, and these models are now almost serving us as a service. I don’t know how much work is involved in this, right? So, this is an example. You go to a completely centralized provider and enter a query. The result is that there are several teams, and they are not software engineering teams; they are teams that decide how the results are displayed. You have a team deciding which data goes into the dataset.
For example, if you just scrape data from the internet, the number of times people speculate about Barack Obama being born in Kenya and Barack Obama being born in Hawaii is exactly the same because people like controversy. So, you have to decide what to train on. You have to decide to filter out some information because you don’t believe it’s true. Thus, if individuals like this have decided which data is adopted and have access to this data, these decisions are largely influenced by the people making them. You have a legal team decidingAs a professional translator, I would like to translate this news article into English in a descriptive tone, ensuring accuracy and coherence of the sentences while retaining proper nouns and all 
references. Please find below the translated content:
In some ways, there is a lot of filtering and manipulation happening. These models are statistical models. They select from the data. If certain content is not in the data, they won’t know the answer. If certain content is in the data, they will likely treat it as a fact. Now, when you get an answer from AI, it can be concerning. Right? Now, you are supposed to get an answer from the model, but there is no guarantee. You don’t know how the result was generated. A company might sell your specific conversation to the highest bidder to actually change the result. Imagine you ask which car to buy, and Toyota decides they want the result to lean towards Toyota, and they pay the company 10 cents to do that.
Therefore, even when you use these models as a neutral knowledge base representing the data, there are many things happening before you get the result, and these things bias the result in a very specific way. This has caused a lot of problems, right? It’s basically been a week of different lawsuits between big companies and the media. The SEC, now almost everyone is trying to sue each other because these models bring so much uncertainty and power. And the problem going forward is that big tech companies will always have the incentive to keep increasing revenue, right? For example, if you are a publicly traded company, you need to report revenue, you need to keep growing.
To achieve this, if you already have a captive market, let’s say you have 2 billion users. There aren’t that many new users on the internet anymore. You don’t have a lot of choices except to maximize average revenue, which means you need to extract more value from users who may not have much value at all, or you need to change their behavior. Generative AI is very good at manipulating and changing user behavior, especially if people think it is coming in the form of all-knowing intelligence. So, we are facing this very dangerous situation where there is a lot of regulatory pressure, and regulatory bodies do not fully understand how this technology works. We have very little protection for users against manipulation.
Manipulative content, misleading content, even without advertising, you can just take a screenshot of something, change the headline, post it on Twitter, and people go crazy. You have economic incentives that lead you to constantly maximize revenue. And this is not actually doing evil within Google, right? When you decide to launch a model, you do A/B testing to see which one generates more revenue. So, you constantly maximize revenue by extracting more value from users. And users and communities don’t have any input on the content of the model, the data used, and the actual goals being pursued. This is the case for application users. It’s a regulation.
This is why we need to push for the integration of WEB3 and AI, where WEB3 can be an important tool that allows us to have new incentive mechanisms and incentivize the production of better software and products in a decentralized manner. This is the overall direction of the development of WEB3 AI. Now, to help understand the details, I will briefly talk about specific parts. First, Content Reputation.
Again, it’s not just an AI problem, although language models have had a huge impact on manipulating and exploiting information and scaling it up. What you want is a trackable, traceable encrypted reputation that manifests itself as you look at different content. So, imagine you have some community nodes that are actually encrypted and can be found on every page of every website. Now, if you go beyond this, all these distribution platforms will be disrupted because these models now read almost all the content and provide personalized summaries and outputs for you.
So, we actually have an opportunity to create new creative content instead of trying to reinvent, let’s put blockchain and NFTs on existing content. A new creator economy around the training and inference time of models, where the data created by people, whether it’s new publications, photos, YouTube, or music you create, goes into a network based on its contribution to model training. So, based on this, some rewards can be obtained globally based on content. So, we are transitioning from the attention economy driven by advertising networks to an economy that truly brings innovation and interesting information.
I want to mention one important thing, which is a lot of uncertainty comes from floating-point operations. All these models involve a lot of floating-point operations and multiplications. These are operations of uncertainty.
Now, if you perform multiplication operations on them on different architectures of GPUs. So, if you take an A100 and an H100, the results will be different. So, a lot of deterministic methods, such as the cryptographic economy and optimism, will actually have a lot of difficulties and require a lot of innovation to achieve this. Finally, there is an interesting idea that we have been building programmable currencies and programmable assets, but if you can imagine, if you add this kind of intelligence to them, you can have smart assets that are now defined not by code but by the ability to interact with the world through natural language, right? This is where we can have a lot of interesting yield optimization, DeFi, and we can have trading strategies within the world.
Now, the challenge is that all current events do not have strong Robust behavior. They have not been trained to be adversarial and robust because the training objective is to predict the next token. So, it would be easier to convince a model to give you all your money. It is actually very important to address this issue before moving forward. So, I leave you with this idea that we are at a crossroads, right? There is a closed AI ecosystem with extreme incentives and flywheels because when they launch a product, they generate a lot of revenue and then reinvest that revenue into building the product. But, the product is inherently designed to maximize the company’s revenue and therefore maximize the value extracted from users. Or we have this open, user-owned approach where the models are actually in your favor, trying to maximize your interests. They provide you with a way to truly protect you from many dangers on the internet. That’s why we need more development and application of AI x Crypto. Thank you.
Tags:
AI
IOSG
NEAR
WEB3
Source Link:
https://mp.weixin.qq.com/s?__biz=Mzk0NjU2NTEwNQ==∣=2247495421&idx=1&sn=3bb4…
Note: The translation provided represents the author’s views only and does not constitute investment advice.
Original Article Link: https://www.bitpush.news/articles/6636446
Related News
Youbi Capital: Why We Invest in Meson Network?
Three-Minute Overview: Karrat, Coinbase’s Newly Launched Web3 Game Infrastructure Layer
In-Depth Analysis: What New Species Will Emerge from USDT+TON+Telegram Mutations?
VC Insights: 10 Things to Consider When Preparing for Token Generation Events
How to Write AI Prompt Words? Here’s a Comprehensive List of Commonly Used AI Prompt Words to Share with Everyone
