.png)
3 months ago
2
Update: This was a fun experiment and I recommend you check out the post for a fun read on setting up reverse shells. But I’ve since discovered this awesome tmate action which lets you interactively debug in the browser or via SSH.
With this step if any previous step in your workflow fails a tmate session will be started and the connection info will be repeatedly printed in the workflow output.
Much easier!
Original post
When a GitHub Actions workflow fails it would be really nice to be able to interactively debug things with a shell. GitHub doesn’t provide anything like a web console or SSH access to workflow runners so in this post we walk talk through throwing shells with netcat and catching them with netcat and ngrok.
Throwing a reverse shell
The most common way to get a shell on a remote system is to log in via SSH. This provides encryption and authentication and makes the whole process simple. However it requires you to run an SSH server on that system and have network and firewall rules configured to allow incoming traffic, and authentication credentials for that system.
Alternatively you can use a reverse shell, which is where a system will connect out to some other machine on the internet and then forward a shell over that connection. This technique is commonly used in the security community to open backdoors in compromised systems, but is also extremely useful for debugging on a restricted environment such as a CI worker.
Catching a shell
In order to “throw” a shell to a remote system you first have to set up a machine to “catch” the connection.
For this example we are going to use netcat to catch our shell, nc is a standard linux utility that is available on most systems.
Now we are listening for incoming connections on port 4444. Beware that this is an unauthenticated and unencrypted connection and we are going to expose it to the internet. For a bit of interactive debugging on open source projects on GitHub this is fine, but this shouldn’t be used for sensitive information or long term solutions.
Forwarding ports with ngrok
I’m also assuming here that the machine you are running this on (my developer laptop in my case) cannot receive traffic on port 4444 via the internet. So we can use ngrok to forward our ports.
Ngrok is a service which allows you to expose ports on your local machine to the internet, for the purposes of developing and testing software.
Once you’ve downloaded and authenticated ngrok you can set up the tunnel.
In this example we can see that port 4444 on localhost is now also available on port 13604 at 2.tcp.ngrok.io. This will be different every time you create a connection.
Configuring GitHub Actions
Now that we are listening for a shell connection we need to add a step to our GitHub Actions workflow to make the outbound connection.
We probably do not want to leak our connection information into our config. It’s ephemeral so it’s not a huge problem, but storing the connection info in a secret is still good practice.
In your repository head to Settings > Secrets and create DEBUG_HOST and DEBUG_PORT secrets with the hostname and port that ngrok gave us.
Then add a last step to your GitHub workflow.
In this last step we use a combination of mkfifo, cat, sh and nc to forward a shell to our remote host.
When your workflow gets to this step it will appear to run indefinitely with no output.
But if we look at the nc session we have running on our local machine we should now see a shell prompt.
We can then run bash here to get a more useful shell.
What can I do here?
Now that you have a shell on your remote system you can do whatever you like. Just be aware that nc will not forward control commands like ctrl+c and will instead close the nc connection. If this happens you will need to restart nc and restart your workflow.
This is also a simple shell so something like SSH which require a pseudo-tty may not work as expected.
But we can still do things like poke around the runner’s system.
And most importantly we can now start debugging our CI steps to see what went wrong.
