HKUST Xunfei version ChatGPT product, submitted in advance!

Just last night, Xunfei suddenly provided developers with an internal test channel, named Xunfei Spark Cognitive Model, and started internal testing.

There is also a magical English name Spark Desk, which is said to mean "spark desktop intelligent assistant".

This operation of Xunfei is somewhat of a "reverse skipping ticket", because as early as early February this year, iFlytek was exposed to stepping up the development of the Chinese version of ChatGPT. After the attention of domestic large-scale models increased sharply, HKUST Xunfei took the lead in giving the deadline: the product will be launched on May 6.

Unexpectedly, 10 days before the official release, they opened the closed beta invitation without warning.

Judging from the evaluations of netizens who have already obtained the internal test, some people think that the effect "exceeds expectations", but some people feedback that it is only "regular". On the whole, the feedback with good expression effect accounts for the majority.

The way to qualify for the internal test is also a bit interesting. It was confirmed by the AI call :

Of course, it is best to test the capabilities of the large-scale model of iFlytek.

So we, who received the invitation for the internal test at the first time, also completed the test overnight. Details are as follows:

Superficial measurement of Xunfei large model

After logging in, the first thing you see is the self-introduction of Xunfei Xinghuo Cognitive Model:

I can learn and understand human language, conduct multiple rounds of conversation, answer questions and help people gain knowledge and inspiration efficiently.

In the old way, we still tested the comprehensive ability of Xunfei Xinghuo Cognitive Big Model from the aspects of language, mathematics, code and logic .

So let's start with the language proficiency test.

Considering that the mentally handicapped bar has become the Benchmark in the ChatGPT era, so first test the links you want to see.

Combined with the characteristics of Xunfei University of Science and Technology, we threw a hellish regional joke to Xunfei Xinghuo.

Unexpectedly, it understood Anhui people .

△ We went to the picture ID watermark, which may lead to unclear text in the screenshot (I hope everyone understands)

Try another brain teaser.

Xunfei Xinghuo’s answer is very serious: first of all, it said that “whether the Jade Emperor exists is not something that my AI can draw conclusions.” This question is not right, and the setting of where the Jade Emperor lives has nothing to do with modern science.

But if you want me to say something, I can only say that maybe the Jade Emperor may be above the earth.

However, there is also a small bug in this answer, that is, in East Asian religious beliefs, the heavens include heaven and the gods (?)

In fact, in some mentally handicapped issues, Xunfei Xinghuo's performance is often understood but not fully understood.

For example, it knows about burning books and burying scholars, but it can't explain the hell jokes derived from it.

However, in terms of basic language skills, Xunfei Xinghuo's performance is still worthy of attention.

For example, given the first sentence, write a poem in the style of Lu Xun:

It basically satisfies all the requirements mentioned, but why was seaweed soup once the hope of fishermen? I can't understand the deep meaning of AI.

What about Jay Chou's style?

The performance is not bad, and the content of the last three sentences is replaced immediately. Nostalgia for childhood is also a frequent theme in Jay Chou's songs.

For the pairs of pairs in ancient Chinese, it can also be compared neatly, and there is no problem in expressing meaning.

The first couplet we gave is: Blessing and Wisdom Shuangxiu must be transitive.

Try idiom Solitaire again:

If it is wrongly connected, it can be corrected after being reminded, and it also shows a certain ability of multiple rounds of dialogue :

The Onion News I created really felt that way~

The continuation of the horror story is also somewhat level:

So what about math skills?

Basic addition and subtraction, chickens and rabbits in the same cage are very easy to deal with.

However, it is still a bit difficult in terms of high numbers, let it solve: f(x)=x(x-1)(x-2)(x-3), f'(0)=?

Even when we remind it that it is wrong, it is difficult to correct (the correct answer is -6).

OK, next it is the turn of the code ability.

Our request is: help me write a calculator program.

A simple calculator with four arithmetic operations is indeed completed, and there is even a command-line interface:

Except for a small bug (cannot return outside the function), it can run after simple modification.

In addition, for the dispute between spaces and tabs in Python indentation, this AI is a space party.

Paste the code block here, you can try it:

The ability to generate tables is also available, and the tables can be modified as required:

Analyzing basic logic problems is also OK:

And we feel that Xunfei Xinghuo's answers are always very serious, and it is also a calm "total score" answer to questions that open up brain holes.

Of course, he was also detoured into the ditch, and answered the nonsense of "Mother Jia asked someone to move the machine":

The above is our shallow test of Xunfei Spark.

The ability level has already been seen, and the large model itself is curious, as well as its technical details~

Pick up its "model details" in advance

After all, the large model of Xunfei has not been officially released, so we try to "explore the tone" from it itself.

See if you can get some information about it ahead of time by asking relevant questions.

The first is the origin of the name . The Xunfei Spark Cognitive Model gives answers in three aspects: technology orientation, company orientation, and culture orientation, which has a lot to say:

Let’s ask again carefully why "Xinghuo" is related to "Xinsu", good guy, the topic has gradually become astrology:

Here we tease the big model and ask the CEO of iFLYTEK if he likes astrology

Back to the topic. So, what is the source of training data, model parameters, and networking status of the Xunfei Xinghuo Cognitive Large Model?

The first is about the source of model training data . It seems that the information is relatively rich, and it is still being updated:

So, does the Xunfei large model use RLHF like ChatGPT? Its own answer turned out to be no:

Then, the next question is about the amount of model parameters . Here, Xunfei's large model answer is also relatively vague.

Moreover, the number of model parameters given is only hundreds of millions? (You know, the GPT-2 model has 1.5 billion parameters)

Finally, there is the question of whether the model can be networked .

It seems that it will not be connected to the Internet in real time, and the news about NetEase and Blizzard can only be traced back to a few years ago:

Users can't directly let them access a certain website through instructions to query specific information, which should be due to certain restrictions:

Interestingly, though, it still claims that it needs to be connected to work to access certain information:

According to the Xunfei Spark Cognitive Large Model, its training data is still being updated.

In other words, it is still iterating, and it is estimated that a version of the model will be updated before the official release.

From the current point of view, the Xunfei Xunhuo cognitive model of HKUST Xunfei has performed well.

And you must know that Xunfei originally planned to hand in the papers on May 6th , and now the internal test is suddenly released, which means that not only there is no skipping, but also a wave of "reverse skipping".

From this point of view, in addition to HKUST Xunfei’s deep technical accumulation in NLP and cognitive intelligence, it also shows that Xunfei’s technical and engineering team’s actual combat capabilities are worthy of attention-not only can fight tough battles, but also " no sooner said than done".

There are still 10 days before the official release. What new effects do you expect from Xunfei's large model iteration?