Wang Xiaochuan: The core of large-scale entrepreneurship is to figure out how to match the technology with the product
AI native applications must provide a 10x better experience and make users feel comfortable using them.
In the past eight months, the Baichuan Intelligence team relied on its accumulation of AI technologies such as search and high-quality data processing to rapidly iterate on model size and quality. Wang Xiaochuan, founder and CEO of Baichuan Intelligence, believes that Baichuan should be "half a step slower in its ideals and three steps faster in its implementation."
On December 16, 2023, at the Geek Park Innovation Conference 2024, Wang Xiaochuan further shared new thinking on large models.
Regarding the evolution direction of large models, Wang Xiaochuan believes that the current large models are "learning" but "not thinking". The next direction of model evolution is to combine "learning" and "thinking". When making applications in the era of large models, we must consider the characteristics of large model technology. This is the biggest difference between making products in the past and now.
"In the past when we were working on applications, we always talked about the match between product and market - PMF (Product Market Fit), but apart from product and market, one word was missing: technology."
He believes that the current large-model technology is still far away from AGI. Under this imperfect premise, it is more important to clarify: "What kind of products is such a technology suitable for?" rather than product managers having insight into the market and starting to work on it.
In Wang Xiaochuan's view, under the new development paradigm brought about by large models, the starting point of product managers should be from thinking about product market fit (PMF) to thinking about how to match technology and products, that is, TPF (Technology Product Fit, Technology product matching).
Wang Xiaochuan believes that what makes a large-scale model application successful is that it can first provide an experience that is ten times better than traditional applications, and users can "enjoy using it." To make such an application, the product manager must not only be a fan of large models, but also have traditional product experience and imagination, and be able to figure out what the large models look like.
The following is the transcript of the conversation at Wang Xiaochuan Geek Park Innovation Conference 2024
01 "Learning" and "Thinking" about Large Models
Zhang Peng: You have been to our conference many times. Just now you were listening very carefully to the discussion between two technical experts. They talked about some key technical factors behind the OpenAI incident some time ago, and even mentioned that large models need to be slow. The ability to think, what do you think?
Wang Xiaochuan: Yes, this year we are planning to build a large model to establish the company in April. I mentioned a few keywords. One is called search enhancement, because we need to integrate traditional knowledge. The second one I hope is to use large models to do reinforcement learning. At that time I mention this point because I have seen that the large model itself represents a way of fast thinking. Like people, I will give you the answer as soon as I pat my head, and I can speak with my mouth. This method of learning and applying reasoning has its advantages. For our own shortcomings, using large models as the starting point is definitely not enough. At that time, we believed that reinforcement learning could be of great help. This is an area that has always been of great concern in Baichuan's internal work.
Zhang Peng: Is it just slow thinking?
Wang Xiaochuan: Yes, it means slow thinking. Compared with slow thinking, today’s big model represents fast thinking. Let me share my two opinions: Thinking fast is not even called "thinking", but thinking slowly I think it has more "thinking", represented by the large OpenAI model, it is called "learning" and its source of knowledge I learned it.
I do not emphasize the "thinking" of reasoning. In fact, when you are studying, you may have to think for a long time. This is called thinking. Therefore, Confucius said before that "Learning without thinking will lead to loss, and thinking without learning will lead to peril." "
Projected concretely, the big model is learning. It actually does not think. It is not like humans, who can think back and forth and open up their imagination space to see what system is thinking? OpenAI has just established the company and what DeepMind is doing, such as making AlphaZero and playing games, this is what I am thinking about.
But that is a reinforcement learning setting, called multi-agent confrontation. AlphaZero is not a learning system. It has thrown away all the previous 60 million games (chess training). Instead, it is playing against itself in the game. Find a new understanding, the ultimatum, it is such a "thinking".
After AlphaZero finished thinking, it stopped in place, that is, "almost". It only does specific tasks and cannot expand it to other fields. Therefore, we say that the large model represents "learning" and AlphaZero represents "thinking". The two systems together will be very powerful.
Wang Xiaochuan, founder and CEO of Baichuan Intelligence, and Zhang Peng, founder and president of Geek Park, analyze the "learning" and "thinking" of large models | Geek Park
Zhang Peng: So the next important thing is to truly learn and think, right? Learning and thinking should go together.
Wang Xiaochuan: Yes. To be more specific, the "thinking" scenario we are thinking of is when you ask a large model how to play Go, but it actually cannot play. But you ask, can a player determine whether he has lost or won in Go? The large model can be judged based on its existing knowledge. Even if you say to write a code to determine the winning or losing of Go, a large model can write this code.
You can also ask it to write a code. It can also write out how the state of the chess is transferred after each move, that is, the entire process of playing chess.
So let's imagine that if the large model is strong enough, although it will not play Go directly, it can write the code for a state transition (Transaction Function) such as playing Go, and the code for finally determining the winning or losing of Go. In other words, there is a chance that the large model can write an AlphaGo code and then run it. After running it, you can play chess. This thing has a chance to happen.
Zhang Peng: European and American technologies are still exploring boundaries, which also makes people feel pressured. How do you think this distance can be measured? Can it be shortened? Can you create different value yourself?
Wang Xiaochuan: Before I went to the United States, I said this in Baichuan, "Half a step slow for the ideal, one step fast for the implementation." Later, after I came back from the United States, I folded my ideal in half, and became "One step slower for the ideal, one step faster for the implementation." "It's three quick steps to success."
02 Large model application, China may "jump ahead"
Zhang Peng: How do you understand that one step is slower for ideals and three steps faster for implementation?
Wang Xiaochuan: After contacting them, I think the underlying thinking of both parties is different. OpenAI is originally a non-profit organization that wants to explore the boundaries of AGI, and they really do so.
The last time I talked to them, they wanted to try to connect 10 million GPUs together to create a large enough system. What is the concept of 10 million GPUs? NVIDIA produces one million units a year, GPT-4 is about 25,000 units, and the GPT-3.5 GPU we benchmark today only has 4,000 units. When they think about problems, their starting point is not in the same world as us, so we can't compete with them in this regard.
In this case, both people and companies have to find their own position. In this soil, we have to be confident that we have the opportunity to move faster on the application.
Maybe as our user data set becomes larger and our technology accumulation becomes stronger, our application will be good enough and even be used in the United States. In this case, it does not mean that you must reach the GPT-4, GPT-5, and GPT-6 stages before you have the opportunity to apply it.
Different soils grow different things, and application is one of the strengths of Chinese tradition, and it is also an innovation. I think it is fair. This is also a better opportunity for Chinese companies, especially now that OpenAI is the dominant company in the United States. Application companies have to make applications based on OpenAI's technology. Whatever the technology is made of, you can do it. Such applications.
However, domestic model companies can make applications themselves. This kind of end-to-end coherence gives them the opportunity to launch applications in one field faster than American companies.
Wang Xiaochuan believes that China’s large models may run faster in applications｜Geek Park
Zhang Peng: Sometimes we are definitely willing to pursue something with ideals and a sense of mission. In the big process of AGI, we can join this team. They may be forwards and break through boundaries, but we may be free agents or rearguards. waist, but he is also meaningful in the team. For example, I can implement the technology and turn it into something valuable.
Wang Xiaochuan: There will be derivation at both levels. What Mr. Peng just said is that as a global citizen, as a Chinese company, you have a division of labor and cooperation in the world, instead of dividing one between ourselves and the enemy, leaving only one kind of competition. We respect and pursue their inventions, but we also have our own unique contributions. It’s not just: I think I need myself and the world doesn’t need me.
03 Large model entrepreneurship, technology product matching is the most important
Zhang Peng: Very good, I think very clearly. In this wave of entrepreneurship, I have found a point of reconciliation with myself, that is, how do we become a player in a meaningful game in the world? Not everyone may want it. Be a forward.
That brings us to another issue. Everyone is talking about Super App today, but no one has seen what the Super App will be in the future. Robin Li also said just now that it cannot be determined today. But I think if we want to make a Super App, what starting point do we need? For example, we used to talk about PMF (product market fit). How to do this PMF today?
Wang Xiaochuan: Yes, I am thinking about this, it may be both closer and farther away. The reason for zooming out is that when we imagine reconstructing the original application, such as refactoring WeChat, this thinking angle may limit ourselves at once, so first, you have to zoom out this perspective.
So back to making Super App, a long-term direction is that it represents the fundamental appeal of people. I summarize this fundamental appeal into three key words. People need to have three things: first, they are creative; second, they need to be creative. One needs health, and the third needs happiness.
Health and happiness are very easy to understand, but creativity comes from the fact that in the world, you always hope that your existence will make a difference to the world. You can make a difference in the world, so how can we help you change the world? Categories that can be separated independently.
When we have this vision, how to do health, how to do entertainment, and how to help you get information and make you more creative, in the long run, these three directions will be there. But conversely, we will also throw away some things, such as writing marketing copy and helping you with customer service conversations. In fact, large models are very good at doing these, but I feel that these do not return to the fundamental needs of people, which leads to another trap. Into the original so-called reconstruction logic.
Therefore, after having such three general senses of direction, I will have different ideas. This is a wide-ranging thinking, otherwise I will fall into the pit of competition from big manufacturers.
Second, let's move closer. I want to mention an important word, which is the word "PMF" just mentioned. I want to use a new word to talk about it, because PMF always talks about the relationship between products and markets. Throw it away and it’s just “technology”. Technology, in the era of AI, still has many imperfections and uncertainties, unlike when we were doing Taobao or WeChat before.
I think technology is the bottleneck now, but in fact technical problems can definitely be solved. It just depends on the level of engineers, cost and other issues. Whatever you want can be realized at the engineering level. However, large model technology, including the illusions and timeliness just mentioned, only knows natural language. This technology itself has limitations and imperfections.
So we are still far away from AGI. Precisely because the technology is not perfect, we need to be clear about what products a technology is suitable for, rather than grabbing the market first, looking around the market and then starting to do it. I think this kind of courage It is quite valuable, but I think how to coordinate and implement the first-principle "TP" technology and products is something we need to think about now.
A good example is Character.ai. The founder of Character.ai does not have a product background. He has a very good understanding of technology, especially the algorithms behind the product. He also has the insight that the technology itself is imperfect and may make mistakes. , so he first thought of using it in the entertainment industry. Secondly, the first thing this technology can carry is natural dialogue. It is a character, so make it into a character.
Zhang Peng: In this way, its shortcomings become its characteristics?
Wang Xiaochuan: Let me mention two concepts first. First, we always felt that we were making tools. Tools actually represent a lot of certainty. But this time we are not making tools. This time we are making partners, more like people. The same new species. We humans need to accept its flaws as well as its strengths. People have hallucinations, and I can use them if they have hallucinations, so why can’t I use machines if they have hallucinations?
In the end, one person should match one thing, so when it comes to technical matching, we believe that we need to change our perspective. Instead of looking at it from a tool perspective, we should look at it from a person's perspective. This is one of my ideas.
Zhang Peng: What you just talked about was Technology-Product-Fit (Technology Product Fit), TPF, not the concept of PMF.
Wang Xiaochuan: Yes, we need to have a sufficient understanding of the technology itself and let the technology match related things. This requires product managers, or the company’s No. 1 product manager must have such an understanding of what large models are good at. Not good at anything. This process is about creating people, not tools.
There used to be a story about a king and a painter. The king was blind in one eye and missing a leg, but he was very narcissistic. When he wanted to paint a self-portrait, he invited painters from all over the country to paint and kill each one because he could paint a self-portrait. It's too similar, but it's missing an eye and a leg, which is slandering the image. But if the artist painted the eyes to be piercing and heroic, that would be to deceive the emperor and kill him, so the problem would not be solved. Later, a painter drew a picture of the king hunting. He was standing on a big rock, with one leg curled up and covered up. The king was drawing the bow, and the missing eye happened to be closed. This way the painting was both balanced.
What technology is good at, what it is not good at, and how to match it, this has greater requirements for product managers. I call it TPF.
Zhang Peng: I think the term TPF is very good. TPF seems to be the starting point. If we stand in the future and we want to make Super App, how can we do TPF well? Under what circumstances is it called a good TPF?
Wang Xiaochuan: In the past, product managers mostly wrote a document describing the function definition and requirements. They could draw structural design drawings to show to the boss. This product should look like this, meet the needs of users, and accurately implement the functions of each step.
This is not the case with today's large models. Every time an input is given to a large model, its output is uncertain and cannot be explained in one word or sentence. At this time, it is difficult to explain this matter clearly with a set of deduction rules. The logic is deductive, and it must be dismantled and turned into a bunch of evaluation sets. The product manager's requirement is not only to define the product, but to transform the defined product into a subsequent evaluation set. In other words, what kind of test set should be made for the output of the model under certain inputs.
At this time, the technical counterpart is not the engineering staff, but the algorithm staff. The working habit before the algorithm is that you give me the evaluation set, and I will optimize my algorithm to meet the evaluation set. Whether it is by adjusting prompts, doing SFT, or Post-Train. In this situation, the product manager defines the evaluation set. After the technology obtains the evaluation set, it then looks for a data set or a training set to train the system to meet the evaluation set.
Wang Xiaochuan explains how to set OKR for large models｜Geek Park
Zhang Peng: This is to set OKR for large models.
Wang Xiaochuan: It has a very rigorous mathematical evaluation method. As long as engineers who have worked on algorithms will adapt to this method, and finally let the evaluation set and data speak, internally, this has become a standard working method.
Including search companies, this method is also used. Search is an algorithm-driven product, driven by evaluation sets. However, when the Internet developed to an advanced stage, technology was not a problem, and it was no longer even algorithm-driven, but engineering-driven. Sometimes, this PMF is not wrong, it just lacks a layer of TPF. In the end, you will find that the product is not unable to meet market demand, but it has been iterating and cannot produce a phased product.
Zhang Peng: You have just explained to some extent an issue that I am very concerned about - what is AI-native development. Essentially, it depends on what we are developing. You have to set the evaluation set under the set goals so that the data set can be effectively trained to meet the requirements of the evaluation set. This is your real development engine.
Wang Xiaochuan: This is called AI-Native. If it is AGI-Native, it will deepen the paradigm of AI model capabilities.
04 A good large model application should make users feel comfortable using it.
Zhang Peng: This will indeed put new requirements on product managers. In the past, it was said that PMF was doing very well, and we were aware of it. For example, user usage increased and the user experience was very good. But how to evaluate the TPF now?
Wang Xiaochuan: TPF first has requirements for product managers.
First, the requirements must be converted into a test set. The test set allows technical engineers to find that the "feel" is improving when meeting the requirements. And when the demo is launched, the distribution of needs raised by users is exactly the same as the distribution of the evaluation set raised by the product manager, and the results in the evaluation set can meet the needs of users.
Second, when promoting products, PMF will be mentioned to see whether the distribution of Marketing Fit (market fit) on the market is consistent and whether users are satisfied.
Zhang Peng: If users can use the product you develop very well, should they use it well or use it happily? If you use it well, the number of users will explode and it will become a Super App; if you use it well, you will develop it step by step. Are we going to pursue an explosion? Or should we solve the problems of the few first, and then the problems of the majority, layer by layer?
Wang Xiaochuan: This is not contradictory. First of all, "good" is compared with the original. You can compare how much better you are with yourself. If compared with mature large manufacturers, a 30% or 20% improvement is a huge benefit. But for startups, if it is an AI-Native native application, it must be comfortable to use from the beginning. At least for a specific type of unique needs, users must feel ten times better.
Zhang Peng: It feels ten times better.
Wang Xiaochuan: It’s not just better, but it has to give you a sense of surprise. Today, when choosing a bright model for large models, the experience must be improved tenfold, and the surrounding demand must be improved fivefold or threefold. Only in this way can the crest be raised high enough and then gradually widened. I think if this product doesn't make you happy at first, it's not enough if it's just better than the original.
Zhang Peng: Many people in the audience today are also very concerned about how to participate in this new era promoted by large models. If you want to be a product manager under the new paradigm. How should they set out?
Wang Xiaochuan: Looking at the attributes of the company, one kind of company is going to be end-to-end, and it will make both applications and models. One is a company that pays more attention to applications. It rarely touches models or solves problems with small models. The two types of companies have different paths, but there is one thing that must be done first, which is to "use" and treat yourself as an enthusiastic fan in the era of large models to experience and feel the differences that this model brings to you, so that You wonder about it, feel it, appreciate it. Today you are going to use this model, just like a friend, you can feel what works and what doesn’t.
Zhang Peng: You must first become a super user of a large model.
Wang Xiaochuan: I believe that fans of Geek Park are born with such motivation and curiosity. After you use it, your inspiration will come out and you will know what it is good at doing, which will then become the idea for your subsequent products.
Zhang Peng: Large model technology is still in the process of rising. You must first follow it and get closer to it before you can consider how to apply it.
The company is constantly growing, and you must be constantly recruiting people. When you choose a product manager, what kind of temperament and experience do you pay attention to? Can you open up your selection criteria?
Wang Xiaochuan: Baichuan plans to release a super application next year. We will not talk about experience, but can only talk about some imagination.
We really hope to find people with previous experience. As a product maker, if you don’t have previous experience and ideas, just say I want to start a business. In this case, it is quite difficult to start a business. We will ask you to present the product completely and visually. You can imagine what a large model will look like, and you also have sufficient motivation, curiosity, and imagination.
Wang Xiaochuan explains in detail the “needs and needs” of product managers in the AI era | Geek Park
At the same time, we also hope that you have previous experience in making traditional products. That is to say, I hope to have previous successful experiences, but also be able to break up my own experience to nourish the large model, and also conceive a new look of the large model. At this stage, it is "both necessary and necessary".
Today, the environment in China is different from that in the United States. Both Baichuan and domestic companies are racing against time and cannot give you three or five years to explore.
05 Enjoy the "push-back" feeling of starting a large-scale business
Zhang Peng: If someone has experience in a related field but has no technical ability, can he independently explore the application of large models? For example, you are working hard in the health field. I have accumulated many years of experience in the health field and have the temperament you mentioned. Should I join your company? Or can you explore it yourself after connecting to other people’s models?
Wang Xiaochuan: Everyone can do both paths, and some people will explore on their own. However, during the exploration process, it is very likely that you will find that you can’t walk anymore, and you will feel a sense of powerlessness. In the end, you still need the support of the model.
Therefore, in China today, there is a greater chance of joining a model company, because it is not yet to the point where it is possible to develop applications independently. There are articles on the Internet about adjusting models for applications, but this era has not yet arrived. Within the next two years, it is better to join a company that can provide platform-level support and help you break up and integrate your original experience. This way, the probability of success will be much greater, and it is possible to make a super application. It’s okay to make small applications, but if you want to make big things, you should try to fully interact with the model company.
Zhang Peng: It sounds like they still want me to join Baichuan.
Wang Xiaochuan: It mainly depends on whether you want to be big or small.
Zhang Peng: If you want to be big, you have to go to Baichuan.
Wang Xiaochuan: Yes.
Zhang Peng: In April, everyone couldn’t wait to sleep at night. Now that the big model has been running for 8 months, the initial excitement has almost subsided. Starting a business is difficult. After settling down for a period of time, how is your mentality when starting this business?
Wang Xiaochuan: In the past 8 months, the team has been running very fast and growing rapidly. Now we are in a period of developing large-scale model methodologies. Although we feel that our previous technology, capabilities, products, attention, and experience are sufficient, we still feel that it is not "lightweight" enough when we do it.
In the process of jointly exploring methods for large models, our understanding of how to find the most effective linkage state between models and applications is also constantly improving. I think a good state is: if you see that you were a fool a month ago, then you have made progress again.
When I first started working a few years ago, I iterated at a weekly rate, and I would find that I didn’t have enough ideas. This time (large model startup) we went back to working on a monthly basis, and we were not so agile. After a month, you can see your previous shortcomings during rapid iteration. In order to participate in the era of large models, our management and product managers are very cautious, walking on thin ice and constantly adjusting their original working methods.
Zhang Peng: This is a state that you enjoy very much.
Wang Xiaochuan: Yes, progress is stimulated every day, and I can grow in multiple dimensions. Even if my ideas are half a step ahead, I sometimes find that I have better ideas as I go.
Zhang Peng: I quite understand this situation. In five years, what will this company look like that will make you feel more satisfied? What are the company's goals?
Wang Xiaochuan: We are exploring super applications in the three directions of helping people create, be healthy, and be happy. I hope it will be one to five years. I really don’t dare to think about five years, because after five years, the height of technological development may not be what we can understand now. Every day we technicians lament that there are new papers and developments, and there is a strong feeling "Pushing back feeling".
I hope that in two years, we will prove that large models can be used as super applications. In terms of health, entertainment, and helping people create, it can bring great help or hope to people as it did in the Internet era. People can After experiencing or using it, I have this belief.
In five years' time, we may have a completely new way of playing. Maybe in five years' time, robots will be running on the ground, everyone will be wearing VR glasses, and everyone's digital clone will come out. Five years is too long, and I am very satisfied if I can think of the scenes of two years.