I have never expected this from GitHub Copilot

Introduction

GitHub copilot is a revolutionary AI-powered tool that assists developers in writing code quickly and efficiently. It can predict my thought process and help me write code in a matter of just a few seconds. However, sometimes Copilot can come up with solutions that are not optimal or even incorrect.

I honestly didn't know what to expect from Copilot when I first heard about it, but after using it for 2 months, I was pleasantly surprised by its capabilities.

In this article, we will answer some of the questions that people are interested in and take a look at some situations where Copilot is our best buddy and where it falls short.

Common questions

Here are some answers to common questions asked about GitHub Copilot:

What is GitHub Copilot?

GitHub Copilot is an AI-powered tool that assists developers in writing code by predicting what they might want to write. It was developed by Microsoft company and uses machine learning to analyze your code and suggest code snippets and even entire functions.

Does Copilot use OpenAI?

Yes, GitHub copilot uses OpenAI Codex, which is a language model that uses deep-learning. The advantage of Copilot is that it is specifically trained on code and programming languages, making it much more effective for coding tasks rather than general-purpose AI models like ChatGPT.

Since GitHub hosts numerous repositories, it has a vast amount of code to train Copilot on. Every month, Copilot is getting smarter and more capable, assisting developers in writing code.

Is Copilot a paid service?

GitHub Copilot is a paid service, but it offers a free trial period for users to test out its capabilities before committing to a paid subscription. To get information about prices and subscription plans, you can visit the official GitHub Copilot webpage.

What editors can use Copilot?

GitHub Copilot is widely supported in numerous code editors and IDEs. Currently, it is available as a plugin for Visual Studio Code and JetBrains' products, such as PhpStorm, WebStorm, PyCharm, and IntelliJ IDEA.

However, Copilot's capabilities are not limited to these editors only. You can find Copilot integrations for Brackets, Sublime Text, Xcode, Android Studio, Notepad++, and so much more.

Can Copilot write tests?

Now, this is why I've decided to write an article about Copilot. Yes, GitHub Copilot can definitely help you with tests. This is where Copilot shines the most. When writing tests, developers often need to create boilerplate code and write repetitive test cases. But with Copilot, you can simply write the test name and let it generate a test function with all the necessary logic.

When it comes to generating repetitive code and test cases, Copilot can be a huge time-saver. I want to show you one example of how Copilot assists in tests.

I was working on a Laravel app the other day. Like I always do, I've created a test file to test keywords' logic for posts. The idea is simple, there is a table of keywords, a table of posts and a pivot table keyword_post, which connects keywords and posts.

Each post can have multiple keywords associated with it. Instead of writing out all the test cases manually, I decided to see what the Copilot could come up with.

This is the test case that I've written by myself:

public function testAddingTheSameKeywordConnectsPostOnlyTheFirstTime(): void
{
    /** @var Post $post */
    $post = Post::factory()->create();
    Keyword::factory()->create(['name' => 'anna']);

    $this->actingAs($this->admin, 'admin')
        ->json('post', self::STORE_URI, ['name' => 'anna', 'post_id' => $post->id])
        ->assertOk();

    $this->actingAs($this->admin, 'admin')
        ->json('post', self::STORE_URI, ['name' => 'anna', 'post_id' => $post->id])
        ->assertOk();

    $this->assertDatabaseCount('keyword_post', 1);
}

When we try to attach the same keyword to the same post twice, it should make only one connection between post and keyword instead of two. This is precisely what the test does. Now, let's see what test Copilot will suggest.

Wow, it suggests me to test that deleting keyword deletes the post connection with the keyword. Awesome, this is a great test to do, I didn't even think about this one. I will accept the suggestion and run the Copilot's test to check if it passes.

It does because in my relationships I have ON DELETE CASCADE option.

Tip. ON DELETE CASCADE clause in MySQL ensures that when a row from the parent table is deleted, all corresponding rows from the child table are also deleted.

Great, the only thing that Copilot didn't guess, is that instead of assertDatabaseCount('keyword_post', 0) should be assertDatabaseEmpty('keyword_post'). It is just a cleaner way to make sure that database table doesn't have any records. That's the only change that I need to apply to Copilot's proposed test.

The next Copilot's suggestion was this:

Testing that deleting a keyword deletes the keyword from the database. I could accept it because it was valid, but in my particular app I don't have a feature to delete a keyword completely, only to detach it from a post. It suggested me this test because I had a written method which was deleting a keyword from a database.

The Copilot was smart enough to check my actual logic and based on my methods it suggested me a relevant test. This is what I had:

public function destroy(Keyword $keyword): void
{
    $keyword->delete();
}

I had a destroy method that is very common in all Laravel apps, which deletes the keyword from the database in my case. The Copilot would have a different suggestion if I had a destroy method written like this:

public function destroy(DestroyRequest $request): void
{
    $post = Post::query()->findOrFail($request->post_id);
    $post->keywords()->detach($request->keyword_id);
}

Overall, it seems like Copilot was able to provide some valuable suggestions for test cases based on your existing code. It's always good to have a second pair of eyes, or in this case, an AI-powered coding assistant, to help catch potential issues and suggest improvements.

Just be sure to thoroughly review and test any suggestions before implementing them in your codebase. The Copilot cannot know things you want to implement that are not already in code. I recommend writing the test name first so that the Copilot knows what you're trying to accomplish and can provide more relevant suggestions like this:

The proposed code by Copilot is perfectly valid, I don't even need to make any changes and tweaks to it.

In some cases, you might want to write a comment to give a Copilot a hint about what you're trying to achieve. Like in this example:

It's a pretty simple example, but trust me, this works even in big codebases. Just remember to use Copilot's suggestions as a tool and not a replacement for your own critical thinking. It supposed to help, not do the work for you.

How can Copilot be improved?

There are 3 things that GitHub Copilot can be improved on:

1. The speed of Copilot's suggestions can be improved;
2. Copilot could be free for open-source projects;
3. AI can be improved over time;

Let's take a look at each of these points in more detail:

1. The speed of Copilot's suggestions can be improved

One area where Copilot could be improved is the speed of its suggestions. I'm pretty sure it will be fixed in the near future, but for now, there are times when you need to wait for a few seconds to see a Copilot suggestion. It's certainly faster than a real developer when you do a pair programming, but improving the speed of suggestions would make Copilot even more efficient and productive.

2. Copilot could be free for open-source projects

Another potential improvement for Copilot could be making it free for open-source projects. I don't know how to do it smart, so that GitHub can detect if a developer is working on an open-source project or personal one, but it would be a great way to support the open-source community.

As we know, when you are working on open-source, you do not get paid a cent. Having the access to something like GitHub Copilot will be extremely beneficial for open-source.

Let's not forget that the company behind Copilot is Microsoft, which is known for its contributions to the open-source community in the last few years. I hope they will consider this improvement in the future.

3. AI can be improved over time

Lastly, as with any AI technology, there is always room for improvement. As we know, the best way to improve AI is to train it with more data. The more time GitHub Copilot has to learn and gather data, the better it will become over time.

As for 2023, suggestions are not quite there yet. You get a feeling that Copilot doesn't understand the context of your app. It would suggest methods that do not exist in objects and classes.

Like here, for example, I define a variable user_email and Copilot suggests the whole user object instead of just an email:

I was expecting Copilot to grab email from the request object, since it's available there.

Nevertheless, Copilot is so frequently updated that you notice improvements almost daily, and I love it. It feels like they are working on it very hard day after day.

Will I continue using Copilot?

I've been heavily using GitHub Copilot for 2 months, since GitHub offers a 2-month free trial. This trial might be removed in the future, so if you're interested in trying out Copilot, it's best to take advantage of this offer today.

My free trial expires in 9 days, and I believe I will pass on GitHub Copilot for now.

There are a couple of reasons for not using GitHub Copilot:

1. The price is not adjusted for other countries;
2. Copilot makes you think less;
3. Suggestions conflict with the editor's built-in suggestions;

1. The price is not adjusted for other countries.

I currently live in Ukraine, and the average salary here is $500/month for 2023. Compared to the US, the average salary there is around $6300/month for 2023. Microsoft should have made the price for poor countries like India, Ukraine, Moldova, etc., much cheaper.

Google is a good example of that. YouTube Premium Subscription has a different price for different countries, and it's much cheaper in countries with lower average salaries.

Two months ago, I left my job and started working on open-source. Like many other developers, I don't have a source of income because the code that I write is usually free for all and available on GitHub. Paying $10 every month is not a big deal when you have a stable job, but for someone who's working on open-source and living in a country with lower average salaries can be a significant difference.

As for 2023, Ukraine is in the war, I know some developers struggle to make ends meet, and for them, even a few dollars can make a big difference. If you want to read about my journey since the beginning of the war in Ukraine, you can check out this post: “What it's like to be a programmer in a country with the war”. It's not a programming related article, but I just wrote it to leave it here for anyone interested.

2. Copilot makes you think less

GitHub Copilot is undoubtedly a powerful tool that can save time and effort in coding. However, it also has the potential to make you think less and rely too heavily on its suggestions. Programming is not just about writing code. It's also about solving problems and understanding the logic behind them. By relying too much on Copilot, you might miss out on the opportunity to learn and strengthen your problem-solving skills.

Why would you think how to write a function when you can just ask a Copilot to do it for you, right? I think that people who are using AI for code completion will be at a huge advantage over those who are not. However, those people who are not using AI will be a lot more challenged and have the opportunity to strengthen their problem-solving skills, which is an essential aspect of programming.

3. Suggestions conflict with the editor's built-in suggestions

This reason is the most annoying one for me. For example, when I'm working with TypeScript and I want to know what properties a particular object has.

type Person = {
    first: string
    last: string
    age: number
    city: string
}

const anna: Person = {
    first: 'Anna',
    last: 'Korotchaeva',
    age: 25,
    city: 'Kharkiv'
}

What I always do is I type anna. and see the list of available properties.

However, when using Copilot, instead of Visual Studio Code's suggestions you might see autocompletion from Copilot.

The good news is that this issue seems to be addressed in the latest releases of Copilot because I've noticed this problem occurring less frequently. I think it will be solved in the following months.

Conclusion

While GitHub Copilot has a lot of potential and is constantly improving, it may not be worth the cost for developers in countries with lower average salaries. If I had a stable income, I would definitely consider using it, but for now, I'll stick to traditional methods of coding and rely on my knowledge and smart IDE intellisense.

That being said, I think Copilot is a game-changing technology and has the potential to revolutionize the coding industry. There is no doubt that every developer will be using it in the future, those who can afford it at least. It's important for companies like Microsoft to consider the economic differences between countries and adjust their prices accordingly.

Overall, I encourage developers to try GitHub Copilot during the 2-month free trial period and decide for themselves if it's worth the cost. As for me, I think I'll wait for a more affordable pricing option before giving it another shot or pay $100 when I have some income. Or, I might change my mind in the near future and pay for a subscription because I'm in love with Copilot for sure.

If you already tried and have an opinion that you want to share, feel free to comment down below, and I would gladly respond. It's important to remember that regardless of what tool we use, it's our skills and knowledge as developers that truly make a difference in this beautiful world. Thank you for making it this far, take care!

Keywords: github, copilot, ai, autocomplete, bot, future, microsoft, helper