Ultimate Test: Are Copilot and Code Whisperer Effective in Software Development?

What are Github Copilot or AWS CodeWhisperer?

These are the two AI-powered code suggestion tools that can recommend code snippets, classes, or even functions. They integrate directly with your IDEs, capturing the context of the code you’re writing and recommending modifications based on that.

Are these worth the hype?

So, to get to that answer, let’s look at some examples of what it can and cannot do.

Deep Dive:

• Setting up a project from zero, making all the models, the controller boilerplate, etc., is a significant problem or time-consuming operation. Let’s see how AI tools were helpful in this.

With just a few comments, as seen in the image, GitHub copilot proposed all the fields with getters, setters, and methods, and instead of writing that piece of code, we can accept and review it once.

• Sometimes, it can be challenging to remember which properties need to be set to ensure proper connectivity between different aspects of a project. This may result in the need to search the web and documentation for guidance. We asked an AI tool to recommend the necessary properties to simplify this process.

• We started receiving recommendations for it by stating my requirements in the comments, and we were able to configure the connections properly within a couple of seconds.

Are these tools fit for generating functions?

• As we are building an employee system, we would require a function to calculate their wage and post it after deducting taxes, which is precisely what we asked the AI tool to generate, and here are the results.

• That’s a fantastic suggestion; we can quickly update the request if we need anything extra.

• It was simple for us to create the CRUD functionality (controller); we had to specify the comments that could be seen in the image. When we changed the getAllEmployee() controller and asked to write the other API after that, it could write the code how we wanted it to. Within 10 minutes, we could generate all the CRUD functionality. The first code generated differed from what we wanted; the try-and-catch conditions needed to be included.

Interesting Outcome:

Regex can be difficult to deal with, but we obtained the regex simply by using prompts with the help of an AI program.

By leaving comments, we could receive ten distinct proposals. After scanning them, we could choose one, and all the pertinent output appeared.

public static boolean checkEmail(String email){
   //create a regex pattern to check if the email is in correct format
   String regex = "^[A-Za-z0-9+_.-]+@sample.com$";
   //create a pattern object
   Pattern pattern = Pattern.compile(regex);
   //create a matcher object
   Matcher matcher = pattern.matcher(email);
   //return true if the email is in correct format
   return matcher.matches();
}

This was the chosen output by avoiding web searches for regex patterns, which saved a great deal of time.

Unit Test Writing with Generative AI:

Although we specified the type of test case that must be produced, we still need to provide the necessary information. As a result, we made the necessary adjustments and initially set up the testing file with the help of some generated boilerplate, but the true magic came later. After that, it behaved in a manner that was described in test cases,

We start receiving better suggestions as we write more and more test cases. Let’s examine a real-world project. Do our actual programs produce the same outcomes?

Feature Shortcomings of The AI tools:

Some features that do not presently exist would make these tools much more helpful, and some of these things have been mentioned in Copilot X; we will see how valuable they are once we get them in hand.

• If you choose a section of code, it could explain what the lines are doing or provide comments for the functionality between the selected text.
• It could offer multiple implementations for logic.
• There is currently no method to ask AI tools to further optimize the code based on memory or time limits; it would be good to see if they can recognize those things and advise the adjustments.
• A chat window should be inside the IDE to address simple questions. ( chat powered aligned on the lines of chatGPT and bard ). That would save a ton of time.

Conclusion:

These tools can code, but you must carefully instruct them with prompts regarding the code you want to generate. They won’t greatly assist you in constructing any required abstract logical systems, but they can create pieces of code you must know how to fit together. Therefore, it is a tool that may aid developers in accelerating the development process but is not intended to take the developer role.

Copilot X is a future product with a built-in conversation based on GPT-4 that can level up the game.

So are there any drawbacks to it?

Yes, there are drawbacks to it.

• These tools sometimes recommend incorrect codes; therefore, they should be considered carefully. Whatever advice they offer needs to be thoroughly examined.
• Because these tools may propose writing outdated code or vulnerable to security problems, developers must be well-versed in technology.
• Sometimes, we need help to get the simple context as well.
• Many popular IDEs do not support them like Eclipse etc.

We are confident that this provided valuable insights into the use of AI tools. Until next time, happy coding!

Chirag Jivani, Lead Product, and Madhur Malpani, Associate Fullstack Engineer at RAAPID, co-authored this article to test the capabilities of AI.