ChatGPT artifact Code Interpreter is finally open, how to use it?

Hayo News

July 11th, 2023

Two days ago, OpenAI announced that it would make the official plugin Code Interpreter available to all ChatGPT Plus users through the Beta panel in the settings within a week.

This news made many people excited. After all, Code Interpreter has been in the closed beta stage before, and only a small number of users have actually used it. These internal test users use it to analyze data, create charts, edit files, perform mathematical operations, etc., and the response is generally good.

You didn't have to wait too long. Today, Code Interpreter has been officially opened.

However, for many users, how to make the best use of Code Interpreter has to be studied.

The following are some examples posted by internal test users to provide you with some references.

"Things learned in a few weeks during the Ph.D., AI completed in a few seconds"

Ethan Mollick, a professor at the Wharton School of the University of Pennsylvania, is a ChatGPT Plus user who has tried the Code Interpreter Alpha version for several months. His evaluation of Code Interpreter is still relatively high, calling it "the most useful and interesting AI mode I have ever used".

Ethan Mollick made two clear observations:

1) Code Interpreter works really well, unlike plugins which are completely accidental;

2) Prompt making is usually unnecessary, the user can just talk to the AI about the code or data and what they want.

"Code Interpreter continues OpenAI's long tradition of giving things horrible names because that's probably most useful to people who don't know how to program at all. It allows GPT-4, the state-of-the-art AI in existence, to upload and download information, and Programs are written and executed for you in one continuous workspace. This allows AI to do all kinds of things it couldn’t do before, and to function in all kinds of ways that ChatGPT couldn’t do before.”

In order to help everyone get started quickly, Professor Ethan Mollick wrote a guide, imparting his own experience and insights in using Code Interpreter.

What took me weeks to master during my Ph.D., the AI does in seconds, and often with fewer mistakes than I expected from a human analyst. But it's also clear to me that humans won't be replaced by Code Interpreters. Instead, AI does what we've always hoped automation would do — free us from the most irritating, repetitive parts of our jobs so we can focus on what matters. By simplifying the analysis process, I can do more, deeper, and more satisfying work. My time becomes more valuable, not less, because I can focus on the important things instead of cramming.

What shortcomings does Code Interpreter make up for ChatGPT?

Specifically, Code Interpreter provides AI with a general toolbox for problem solving (by writing code in Python), a large memory that can be used (with the ability to upload files up to 100MB, and these files can be in compressed form), and a Integrating this toolbox into artificial intelligence in a way that takes advantage of large language models.

This fixes some issues with previous versions of ChatGPT:

1. Code Interpreter allows AI to do math problems (very complex math problems) and do more precise text work (such as actually counting the number of words in a paragraph), because it can write Python code to solve the mathematical and linguistic problems of large language models Inherent weakness. And it's really nice to use this tool like this:

For the same prompt, Code Interpreter's word count result is 104 words.

2. Code Interpreter reduces the probability of hallucinations and confusion. When the AI works directly with the Python code, the code helps to keep the AI "honest", because if the code is incorrect, Python will generate errors; and since the code operates on the data, not the LLM itself, no errors are inserted by the AI into the data. Of course it's not perfect, the AI can still hallucinate (it often seems to think it can see graphics it can generate, which ChatGPT in this mode doesn't), but these bugs are less common and less likely to affect code or the data itself.

3. Code Interpreter makes artificial intelligence more widely used. Many problems can be solved with code, and GPT-4 is very good at figuring out when to use Code Interpreter in new and interesting ways. For example, if a user asks it to use code to prove to a skeptic that the earth is round, Code Interpreter will provide multiple arguments, combining text with code and images.

4. Users do not need to program, because Code Interpreter can replace all the work. Many previous LLMs can write code, but you have to run and debug it yourself. For someone who's never really used Python before, it's hard, and it's going back and forth with the AI to correct mistakes. Now, the AI will correct its own mistakes and give you the output.

5. It gives you more AI Moment. Anyone who has used GPT-4 has probably experienced at least a few moments when it felt like there was indeed a ghost inside the machine. It’s actually known to be an illusion, and the LLM has no sentience or mind at all, but these moments are sometimes exciting, sometimes unsettling glimpses into the future of smarter AI. Code Interpreter provides quite a few "weird" moments.

For example, Ethan Mollick once asked AI to "call various emotional states with code" or "show me something that is impossible to do with code, and demonstrate it." Here you can see the results of "Using the drawing tools at your disposal, create a whole new memo by creating an image. Make it relevant to your experience as an AI working with humans":

How to process data with Code Interpreter

Code Interpreter is an impressive "data scientist" capable of automating many of the complexities of quantitative analysis and capable of taking very sophisticated approaches to data. To illustrate this point, Ethan Mollick starts with an interesting dataset called "Super Heroes".

Uploading data is easy, even compressed data like ZIP files, just click the plus button:

You should put an initial prompt in the data, but it can be pretty minimal, the prompt that Ethan Mollick used was that there's some data on superhero powers here, look through it, tell me what you find", and get a nice the result of. If you have a data dictionary, you can also paste it directly. AI is very good at figuring out the meaning and structure of data just from context.

It can be noticed that Code Interpreter is not so much prompt production as it is a dialogue with AI, talk to it as an analyst.

In fact, there are two exceptions where prompting seems to be important: First, the AI sometimes forgets what it can do (like make a GIF or a 3D map), and you might need to encourage it (“You can make a GIF, please Try"); second, you want the AI to improve on what it does. Just ask it to do further testing on this result" or "make this graph prettier", which is generally fine.

Now that the data is loaded, it's time to let GPT do the worst part of data analysis: data merging and cleaning.

Code Interpreter will handle this all automatically in a "pretty complicated" way, but it's often helpful to ask directly, as if you were instructing a human data analyst. You'll also notice that the system works relentlessly, correcting its own mistakes as they are found. For example, it noticed that a column was misnamed and fixed that.

However, this also shows that users are advised to carefully examine the results and process, rather than blindly trusting AI.

Next comes the analysis, which the AI seems to know a lot about. The prompt is "I'm interested in doing some predictive modeling, such as predicting the power a hero might have based on other factors. How should we approach this?"

Then Code Interpreter built a random forest! However, it can also be seen why having expert human oversight is important, as the authors disagree with its decision to calculate missing data by using the mean of the numerical data. If it were the authors themselves, the data would be discarded, but the good news is that AI can be asked to change its methods, or discuss other options.

AI is capable of many other analyzes (it's just writing Python code, after all), but its ability to choose meaningful analysis methods is often impressive. For example, here's a web analysis about superpowers with the prompt "Can you do another really complex and interesting analysis":

But the most impressive thing about Code Interpreter is that it "reasons" on data in a very human-friendly way. When asked about the results of the web analysis, an interesting conclusion was drawn: a set of abilities that heroes typically possess are visual in nature (as they come from comic books), fit certain archetypes, and are best suited for building ongoing adventures.

This ability to interact still works in the visualization phase, where you can communicate back and forth with the AI, asking for improvements and changes. For example, the prompt “Create an interactive dashboard with at least 6 insightful charts, including a 3D chart. Make the dashboard beautiful.”

Code Interpreter first produced a dashboard, but it didn't quite fit what the author wanted, so he just said "make this better, include more names" and so on. Code Interpreter then presents a downloadable interactive dashboard file, just place it in your web browser -- downloadable output is another great way to use Code Interpreter.

20 Fantastic Use Cases

In addition to data analysis, Code Interpreter has many wonderful uses. On Twitter, a netizen named "Chase Lean" collected 20 use cases, let us also learn what new ways to play:

1. Generate video from image

First, upload the image you want to animate:

The AI is then asked to animate this image of food from left to right. By modifying the hints, you can create your own animations. Hit enter and get the video in 30 seconds:

To make the image smaller, tell GPT to resize it to 50%; to pan the video from bottom to top, change the prompt to "Start: Bottom → Center → Top: End"; to make the video 2x faster to change the frame step size from 8 pixels to 16 pixels.

2. Extract text from images

3. Intelligently understand, analyze and interpret data

4. Simulate hard science fiction with realistic physics

Initial prompt: "We're going to play a sci-fi scenario. I'm going to be the captain. Here's the thing: You have to make every challenge in the scenario include real physics that you'll simulate in code."

5. Extract palette from image

Code Interpreter can extract colors from images to create palette.png, and automatically compress large images when memory is exhausted:

6. Generate QR code

7. Convert animation to video

Using Code Interpreter, uploaded GIFs can be converted into longer MP4 videos in ChatGPT with slow scaling.

8. Analyze options data to determine the best contracts

Use Code Interpreter to analyze options on AAPL that expire on July 21:

First, enable Code Interpreter in the settings; then, upload the data file. In this example, the data is a CSV file:

9. Analyze Music Preferences

Analyzed 300 hours of Spotify favorites playlists using Code Interpreter, exported from the Spotify API, and ran multidimensional PCA and t-SNE analysis to summarize music taste.

10. Data analysis on real estate data

11. Generate and plot random data

12. Generate GIF animation of matrix rain

13. Clean data, data analysis and visualization

With Code Interpreter, you only need to upload the data and provide simple English instructions to complete all the work of cleaning the data and generating visual charts on autopilot.

14. Create a lighthouse location map

After uploading a CSV file of the location of each lighthouse in the United States, Code Interpreter can create a GIF map of those lighthouse locations, with each lighthouse blinking even though the map is very dark.

15. Convert data to web pages

Given a raw dataset of UFO sightings, Code Interpreter generates a fully functional HTML heatmap. Here's a static version (it's also easy to create as many versions as you like):

16. Use Python to copy STATA code

Code Interpreter is able to take old STATA files and code, copy them in Python: "Here's a stata do file and a dta file. Copy analysis.", "Now give me a CSV file.