Efficient Remote Linux Development

Linux is playing an important role in tech business. It is used as backend servers for most of the large tech companies. Linux development is an important skill if you are software engineer.

However, setting up an efficient remote Linux development environment is not straightforward. This article gives it a brief introduction.

The following figure is an overview of the remote Linux development environment.

Tools

There are many tools involved in remote Linux development. They are listed in the following sub-sections.

terminal

To set up your dev environment in terminal, the first thing you need is a terminal. Technically, it should be called as a terminal emulator, but I’m going to call it a terminal for short as most people do.

If you don’t know what a terminal is, you can think of it as a graphical interface to interact with your terminal applications.

There are many options for a terminal, e.g., iterm2 on mac, mintty on windows, xterm on Linux. Recent years terminals are packing with more and more fancy features. For example, here are some features provided by iterm2.

You can pick up any terminal as you prefer, but I highly recommend that you take the following as a priority:

• The terminal has to either provide a relatively full set of escape codes or a way to allow you to define custom escape codes. This is extremely important, because in terminal, hotkeys are delivered using escape codes, and you don’t have enough escape codes, you won’t be able to use hotkeys.
• Hopefully it supports true color. This is nice to have, but not mandatory.

Personally, I use iterm2 for my daily work.

To learn more about terminal, check out this article.

ssh client

If you are using mac or Linux as your client OS, ssh already comes with them. If you are using windows, you can install cygwin.

ssh (here by ssh, I mean ssh client) is needed if you are doing remote development. It’s a relatively simple tool, which streams your commands from your client side to the server side, and also propogates back the output from the server side to the client side.

ssh is cool, but there is two major caveats:

• If your client side OS is down for a while, you will need to set up another ssh session after you wake it up.
• If your network has a large delay, you’ll see lags when you are typing in terminal.

Unlike ssh, a tcp based tool, mosh is udp based, and perfectly addressed the above problems.

Personally, I use mosh for my daily work.

ssh server

ssh server is included on most Linux server by default. It forwards the commands from ssh server to the program running in the ssh session, which is tmux, in our case.

tmux

tmux might be the most important tool on Linux in term of programming. It addresses two major pain points of a direct ssh or mosh session:

• In each session, you are only allowed to run one single foreground program at any time. But you need more (editor, shell to run the programm, debugger, perhaps a db shell, etc).
• If your ssh or mosh session is down, you lose the foreground program in that session. This is not desirable because you might want to run some long running program, e.g., training a machine learning model.

tmux addressed both above pain points by building a virtual terminal on the server side, and it provides window mechanisms to manage your programs.

shell

Shells are used to issue commands. They are executed within tmux in the above dev environment.

I personally use bash, but you can use any shell you prefer.

text editor

Within tmux, you can also run other applications, e.g., text editor. Text editing is very important in the dev environment. It’s really nice to have text edited on the server directly.

The best text editor within a terminal is emacs or vim. Either one is pretty powerful. Both of them have a backend script langauge, and you can use them to define any text editing features you want.

I personally use emacs more, but you can definitely use vim if you want.

Example Snapshot

Actually, this article is written within such a remote Linux dev environment. Here’s a snapshot of it.

The window you see here is an iterm2. Within it, a ssh session is connected to my Linux server. Within the session, I run a tmux. Inside of tmux, I have three windows (three tabs on the bottom).

• In the first tab, I’m using emacs to write this article.
• In the second one, I’m using git to submit the change.
• In the third one, I’m using a docker to host the website to show my local changes.