Another Look at GPT-3 / Codex / GitHub Copilot – I Have Mixed Opinions

GPT-3 (“Generative Pre-trained Transformer”) is a large language model that can generate text, such as a response to, “Write two paragraphs about the history of computer programming.” GPT3 was trained on an enormous corpus of text — Wikipedia, books, blogs, and so on. GPT3 is owned by OpenAI with major funding by Microsoft. ChatGPT is a chatbot based on GPT-3.

Codex is an extension of GPT-3 and ChatGPT where additional training on billions of lines of computer code was applied — so in addition to natural language, Codex understands natural language and computer code in several languages such as Python, JavaScript, C#, C++, HTML, CSS, SQL, and others.

You use GPT-3 and Codex by typing queries into a text box. A Codex query might be, “Show me a C# function for binary search.” The result would be code that you can copy-paste.

GitHub Copilot is essentially a wrapper over Codex that’s integrated into a development environment such as Microsoft Visual Studio. This is more efficient because you get the Codex results directly into a program instead of having to copy-paste. A downside is that, after a free trial period of 60 days, you must pay $10 per month for Copilot (currently — by the time you read this he cost might be different).

A long time ago, I experimented with a pre-release version of Copilot. It had quite a few bugs and glitches. I figured I’d investigate the released version.

Because I use Visual Studio for most of my C# development, I decided to install Copilot on VS. Installation directions for Copilot at https://docs.github.com/en/copilot/quickstart were very good and I had no major trouble getting Copilot installed. Briefly, I logged onto my GitHub account, went to Settings, and requested access to Copilot. After giving my credit card info, GitHub asked for an authorization code. I opened up VS and created a dummy project and then added the GitHub Copilot Extension from the Microsoft Store (rather than from GitHub as I had first expected, because there is billing involved). VS then gave me an 8-character authorization code which I placed into GitHub. I restarted VS and viola, Copilot was installed in VS.

Using Copilot in VS is quite a remarkable experience. Briefly, you enter a comment and then hit enter. I used a dummy Console Application (shell) C# project.

I typed the comment:

// show code for a list of color items

and got:

public class ColorItem
{
  public string Color { get; set; }
  public string Name { get; set; }
}

Cool. Then I typed:

// create a list of color items

I got the following. Notice Copilot knew the current conversation context.

List colorItems = new List
{
  new ColorItem { Color = "Red", Name = "Red" },
  new ColorItem { Color = "Green", Name = "Green" },
  new ColorItem { Color = "Blue", Name = "Blue" }
};

Next I typed:

// now function to print the list

and got:

public void PrintColorItems()
{
  foreach (var item in colorItems)
  {
    Console.WriteLine(item.Color + " " + item.Name);
  }
}

I have strong mixed opinions about Codex / Copilot. Copilot is essentially an efficient alternative to issuing queries into Google and then reading through blog posts, Stack Overflow answers, and so on. But the inefficiency of the Google approach is useful because it simultaneously teaches me and gets lodged into my memory. Copilot just gives an answer with no active learning going on and so my skills don’t grow.

My conclusion is that Copilot (and Codex) is probably good for simple programming tasks such as generating CSS, SQL, or common C# code like an array binary search function. But Copilot is probably not good for advanced algorithms such as those I use in my machine learning work.

---

Two parodies of computer programming book covers that are spot-on.

---