Exploring GitHub Copilot for CLI

Shell supercharged by natural language. Its installation, usage and current limitations.

Hello and welcome to Behind the Mutex! Check out our latest posts:

REVIEW: LangChain

IDEA: More than Internet Movie Database

#3 WEEKLY: JS/TS, DevOps

Probably like many others we recently got our hands on the technical preview of GitHub Copilot for CLI. To get access, you can sign-up for the waitlist following the link.

So what is it? GitHub Copilot for CLI is a tool that enables users to interact with their command line via natural language. Instead of trying to remember certain commands and their arguments users can now express their needs in a free-form text request.

Given the request, the tool suggest a command and explains the meaning of all the suggested arguments and subcommands. The user then decides whether the suggestion is what they’ve been actually looking for, or whether they want to adjust the request.

Let’s start with the installation process.

GitHub Copilot for CLI is available as an NPM package. It requires Node.JS 16 or higher. You should already have access to GitHub Copilot and be notified about your access to its CLI counterpart.

# Install the package globally
npm install -g @githubnext/github-copilot-cli
# Interactively authenticate the tool with GitHub
github-copilot-cli auth
# Install handy bundled aliases
echo 'eval "$(github-copilot-cli alias -- "$0")"' >> ~/.zshrc

GitHub Copilot for CLI comes with a set of aliases that translate natural language into shell commands:

• ?? - generic requests

• git? - Git-specific requests

• gh? - GitHub CLI-specific requests

As part of experimenting with the tool, we decided to pick a popular public GitHub repo and then explore its contents using Copilot. We chose MLFlow that has 14.2k stars.

As you can see above, an error was reported. Since this is a technical preview, things are not expected to be ideal. Note thought that the tool did not fail and indeed suggested a command to execute.

The suggestion is partially correct as we do want to clone a repository and the explanation states the command’s intent clearly. What the tool did not get right is which repository to clone. Obviously, given the mlflow string, choices are ambiguous, but here we thought that Copilot could’ve gone deeper by asking for specifics.

Below the Explanation section goes the menu with actions that the user can take. Usual arrow and j/k keys work as expected.

📝 Revise query seemed to be a good option to provide specifics and further iterate on the command.

But unfortunately it did not work either. Anyways this is perfectly fine as our request might be considered ambiguous after all.

Still, if we ✅ Run this command, the tool will ask for a confirmation:

  This will execute the suggested command in your shell.
  Are you sure? (y/N)

And as expected, git clone just outputs its usage information. This is not what we wanted.

As mentioned previously, the package also offers an alias specifically for git commands: git?. Let’s try to perform the same request using it.

We see the same situation here, although we even added more specifics and our request almost literally resembles the command that should be suggested by Copilot. Cloning as always.

git clone git@github.com:mlflow/mlflow.git
Cloning into 'mlflow'...
remote: Enumerating objects: 73459, done.
remote: Counting objects: 100% (577/577), done.
remote: Compressing objects: 100% (417/417), done.
remote: Total 73459 (delta 306), reused 337 (delta 148), pack-reused 72882
Receiving objects: 100% (73459/73459), 139.10 MiB | 10.08 MiB/s, done.
Resolving deltas: 100% (55019/55019), done.

Having cloned the repo, let’s find all python modules that import the os module from the Python standard library:

The resulting explanation is pretty extensive. It really offers a great opportunity to learn and memorize various command options and arguments. After the confirmation the command does output the expected result.

One interesting feature worth mentioning is that the tool leaves the executed commands in your shell’s history along with the invocation of the tool itself:

Now let’s get back to our example with the Python modules. You might have noticed that Copilot suggested using grep, which is a reasonable choice. But when it comes to searching files, many would use a much faster tool like ripgrep and the likes. And ripgrep was available on the machine where Copilot was tested. How about we ask the tool to suggest the command accordingly:

And here we go. The fastest tool available happens to be ripgrep according to Copilot for CLI. And we even avoided piping commands for further filtering.

How about requesting something more complex? Let’s try to create a network namespace and enter it … on macOS which does not have a notion of network namespaces (although one can create a virtual network interface there):

GitHub Copilot for CLI suggests a sequence of commands for this request. Two points to note:

1. The suggestions assume presence of the ip tool which is not the case on macOS

2. The sequence of command does not terminate on failures which may not be what the user wants

And finally, let’s still try to enter a network namespace, but from within a container in a Linux VM:

Similarly to our previous experiments, the tool produces quite informative and verbose explanations. But the actual commands won’t work. Things to note:

1. The resulting commands are to be executed within the same shell, but the idea is to execute ip and ping from within the container.

2. There was no suggestion as to install the missing iproute2 package.

3. The ping invocation somehow ignored the address specified in the request

Overall, GitHub Copilot for CLI seems to be an impressive tool with great potential. Users can benefit both from learning commands and detailed explanations and from actually running the suggestions if those look correct.

Most of the shortcomings of the examples above take place because the current iteration of the technical preview may not make assumptions about the environment it runs in, including the underlying OS and availability of certain tools. We believe that further iterations may transcend the functionality to the whole new level.

---

If you have any feedback or would like to discuss things further or share your thoughts, please feel free to comment, send an email or DM the author on Twitter @dalazx.

---