Introduction
In 2016, Microsoft launched Windows Subsystem for Linux (WSL) which brought robust unix functionality to Windows. In May 2019, Microsoft announced the release of WSL 2 which includes an updated architecture that improved many aspects of WSL - especially file system performance. I have been following WSL for a while but now that WSL 2 is nearing general release, I decided to install it and try it out. In the few days I have been using it, I have really enjoyed the experience. The combo of using Windows 10 and a full Linux distro like Ubuntu is a really powerful development solution that works surprisingly well.
The rest of this article will discuss:
- What is WSL and why you may want to install and use it on your system?
- Instructions for installing WSL 2 and some helper apps to make development more streamlined.
- How to use this new capability to work effectively with python in a combined Windows and Linux environment.
What is WSL?
One of the biggest issues I have had with Windows in the past is that working from the command line was painful - at best. The old Windows command prompt was no match for the power available with a simple bash shell and the full suite of unix commands. WSL fixes this in many ways. With WSL, you can install a real Linux distribution on your Windows system and run it at close to bare metal speeds. You can get the best of both worlds - full unix support in parallel with MS Office and other Windows productivity tools not available on Linux.
The concept may be a little tough to grasp at first. Here is a screenshot to bring it into a little more perspective:
In this screenshot, I am running a full version of Ubuntu 18.04 on Windows alongside Excel and Word. It all runs at very acceptable speeds on my laptop.
There have been virtualization options such as VMWare and VirtualBox for a while. The main advantage of WSL 2 is the efficient use of system resources. Microsoft accomplishes this by running a very minimal subset of Hyper-V features and only using minimal resources when not running. With this architecture you can spin up your virtual Linux image in a second or so and get started with your Linux environment in a seamless manner.
The other benefit of this arrangement is that you can easily copy files between the virtual environment and your base Windows system. There are also some cool tricks to seamlessly use Visual Studio Code and Windows Explorer to bridge the gap between the two environments. In practice, it works very well.
I will go through some additional examples later in this article and highlight how to do python development in the various environments.
Setting Up WSL 2
I highly recommend using WSL 2 due to the speed improvements with the file system. As of this writing, these instructions are the high level process I used to get it installed on my version of Windows 10 Pro. I recommend checking out the official Windows documentation for the latest instructions. I also found this article and the official Ubuntu WSL page very useful for getting everything set up.
I will apologize in advance because this article has a lot of images and is pretty long. However, I wanted to develop a fairly complete guide to bring together many ideas into one place. I hope you find it useful.
With that caveat out of the way, let’s get started.
Before you start, make sure you have administrator access on your system. You also need to be part of the Windows Insider program in order to get WSL 2. This may change in the future but for now, make sure you are enrolled. I chose to use the “slow” ring as shown below:
In addition, you need a Windows 10 Version of at least build 18917. I am using Windows Pro but I believe the Home edition will work as well.
If these are new settings for your system, make sure all the updates are applied before proceeding.
Now that the foundation is setup, you need to enable the Windows Subsystem for Linux and Virtual Machine Platform using these PowerShell commands:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
Check the settings here:
You should restart to make sure the install is complete.
Now that the subsystem is installed, you need to install your preferred Linux distribution from the Microsoft Store. I have chosen to use Ubuntu. There are a few tweaks to this Ubuntu distro to make this combo work better so I recommend Ubuntu as the first distribution to start. A benefit is that once you get Ubuntu working you can install other distributions side by side and experiment with the one that works best for you.
The install should not take long. Once it is done, you should see an Ubuntu item on your Windows start menu. Go ahead and click it. You will get a message that the install make take a few minutes:
Then configure your username and password:
It’s always a good idea to update your Linux environment using sudo :
sudo apt update
sudo apt upgrade
As you can see, this is just like the normal Ubuntu upgrade process but on your Windows system.
Go ahead and try some of your favorite commands and watch them work. Pretty cool.
The final step is to use the Windows wsl commands to enable WSL 2 for this virtual environment. You need to invoke the wsl commands from PowerShell as an administrator:
The wsl command is used to manage the different environments installed on your system. Use the command wsl -l -v to see what you have installed:
As you can see from the output, the Ubuntu-18.04 version is still at Version 1 of WSL. We want to upgrade, so use the command, wsl --set-version Ubuntu-18.04 2
Behind the scenes, this process is upgrading the environment while preserving all the existing configurations. It may take a couple of minutes for the upgrade to complete. If you are interested, this link provides some more detail on the differences between WSL 1 and 2.
When you are done, use wsl -l -v to verify that both are running Version 2.
While you are at it, you should probably use this command to set WSL to use version 2 as the default for all new installs - wsl --set-default-version 2
Note, depending on when you install this, you could get a message that says “WSL2 requires and update to its kernel component.” If you see this, reference the info in this MS blog post.
At this point, we have WSL version 2 up and running. Before we go into using it with python, I want to install a couple of additional components to make the development process easier.
Helper Apps
Windows Terminal
One of the problems with the default Windows environment is that there is not a good Terminal application. As I mentioned in the beginning of this post, working from the command line in Linux has been so much more useful than on Windows. Fortunately, Microsoft has been developing a new Windows Terminal that works really well with WSL and the other consoles. If you are interested in learning about all the differences, I highly recommend this blog post for more detail.
The bottom line is that I recommend installing the Windows Terminal from the Microsoft Store. I will be using it for the rest of the command line examples:
Windows Terminal is very configurable and you can trick it out quite a bit. In the interest of keeping this post manageable, I’ll refer you to another post that has some good details and links.
I have configured my terminal to launch a couple of different shells. I found the process of editing and configuring much simpler than the steps I have to do to setup a Windows Shortcut to launch conda.
If you want to review my profile.json file, I have placed a copy on github. I included commands to launch miniconda and customized some aspects of the prompts. Feel free to use this as a reference but you will need to customize it to work for your system. If you have favorite tips and tricks, include them in the comments.
You will need to create your own GUIDs for the different sections of the profile. One option is to use python.
import uuid
uuid.uuid4()
One final item you should consider is installing the Cascadia Fonts for a nicer looking Terminal experience.
After configuring, I have a single place to launch all the various shells and environments I might need both in Linux and Windows:
Miniconda
As you can see from this screen, I have also installed Miniconda on my system. In a fun twist, I installed a version on the Ubuntu image as well as on Windows. I will not go into the process for installing but I do encourage you to install it on your system in your Windows and WSL environments. This will be the default python environment setup I use.
VS Code
The final useful component is Visual Studio Code and some helpful extensions. I recommend that you install Visual Studio Code in your windows environment.
In order to get the maximum usefulness out of this setup, you need to install a couple of extensions:
You will likely want to customize other aspects with themes and icons which I encourage you to do. The above mentioned extensions are the important ones for working with WSL and conda environments on your local Windows and Ubuntu environments.
Working Across environments
Accessing files
That was a lot of setup! Now what?
You should be able to launch your Ubuntu environment or Windows environment and work with Python like you normally would.
Here is a screenshot showing one terminal with tabs running Ubuntu and PowerShell and another one running conda on the Windows system:
This in an of itself is pretty useful but the real power is the way you can interact across WSL and native Windows.
For instance, if you type explorer.exe . in your Ubuntu environment, windows will launch the Explorer and show the current directory in the WSL environment:
Now you have a Windows Explorer view of the files in that Ubuntu WSL environment.
You can also access this environment directly in Explorer by typing the network path \\wsl$\Ubuntu\home\chris
This cross-environment “magic” is supported by the 9P protocol file server which you can see referenced via the mount command in the screen shot above. Microsoft has a nice write up on their blog with some more details about how this works.
Visual Studio Code
There is another handy tricky for working across environments. You can use the WSL Visual Studio Code plugin to access the WSL file system from your VS Code install on Windows.
If you execute the command code . in your Ubuntu environment, Windows will Launch VS Code and connect to the files within WSL. You can edit those files using all the normal VS Code functionality and all changes are saved within the WSL environment. You can see the indicator in the bottom left that lets you know you’re interacting with WSL and not the standard Windows system.
You can also launch VS Code in Windows and access all your running WSL environments through the command palette. Press Ctrl + Shift + P then type Remote-WSL to see the options.
If you have more than one WSL environment setup, then you can select the appropriate one as well.
VS Code then ensures that you are editing files in the WSL environment. For instance, When you open a file, you will only see the WSL file system:
One minor surprise I encountered is that you need to make sure any VS Code plugins you want to use in WSL are installed within the WSL environment. For instance, if you look at this screenshot, you can see how some of the plugins are installed on the local Windows environment but you need to also ensure they are installed on the WSL environment as well.
Fortunately, the installation is pretty simple. In fact, VS Code prompts you with a button that says “Install in WSL: Ubuntu.” The install process is simple but it is an implementation detail to keep in mind.
It is a bit crazy to think about how this works but the implementation is very seamless and in my experience you get used to it pretty quickly.
Jupyter Notebooks
Another method to work across environments is using the network. In researching this article, I found many comments that accessing localhost did not work in some of the older versions of WSL. I did not have any problems getting localhost to work when using Pelican or Jupyter Notebooks. I get the sense that this is an active area of focus for the developers so keep this in mind as you experiment on your own.
The one option I do recommend you use is the --no-browser switch to avoid a warning message when launching a Jupyter notebook. In the example below, I’m running Jupyter in Ubuntu but viewing it on my local Edge browser.
Also, it’s helpful to remember that if you want to copy data from the terminal use Ctrl + Shift + C to copy and Ctrl + Shift + V to paste. You will likely need this to copy the token and authenticate with the Jupyter process.
Directly running apps
The wsl command is a powerful tool for operating on WSL environments. One of its capabilities is that it can run an executable directly from the Linux environment. For example, we can run the fortune command that is installed in our Ubuntu environment.
What about running Graphical Apps?
For the most part, I’ve been using the Windows-native apps for graphical applications. Between MS Office Apps, Chrome and VS Code I have most use cases covered. If I want to use apps like Gimp or Inkscape, I can use the Windows versions.
However, I have found a couple of niche apps that did not have a good equivalent in Windows. One simple app I use is Trimage to compress images.
I can install it in Ubuntu, but when I try to execute it, I get an error message, trimage.py: cannot connect to X server
The fix for this is to install an X Server on Windows. There are several options including a pay version called X410. I elected to use VcXsrv (oh sourceforge, such memories).
Be forewarned it is definitely not a Native Win 10 app so this is all going to look a bit ugly. There are likely ways to make it look better but I have not investigated because this is an approach of last resort for a handful of apps. I’m sharing for completeness.
Anyway, Install VcXsrv and run it:
I needed to Disable Access Control:
Once it is launched, it will sit in the system tray and listen for connections.
In order to configure your Ubuntu environment, make sure these two lines are in your .bashrc file. After making the changes, restart the shell:
export DISPLAY=$(awk '/nameserver / {print $2; exit}' /etc/resolv.conf 2>/dev/null):0 # in WSL 2
export LIBGL_ALWAYS_INDIRECT=1
If all is configured correctly, you should see Trimage:
It worked just fine for compressing the images in this post.
If you really feel the need to run a more full-fledged graphical environment, you can install a light-weight desktop environment like xfce and launch it as well Here is how to install it:
sudo apt install xfce4
Here is a busy screen shot showing all of this working together:
The image shows:
- VS Code, editing files in the WSL environment
- A full xfce desktop running on WSL being displayed in the local Windows X server
- The WSL environment serving up the pelican blog
Workflow
Now that you have all these options for python development on a single machine, you need to decide how best to configure for your needs. I am still working through my process but here is what I’m doing right now:
- Chrome on Win 10: General web browsing, email, Jupyter notebooks
- Visual Studio Code on Win 10: Text and python file editing
- Visual Studio on Win 10 connected through WSL: Write ReStructured Text articles for blog
- Ubuntu on WSL: Maintain and develop content on Pelican blog
- Ubuntu on WSL: Command line tools as needed
- python on WSL: Blog content and general development/experimentation
- python on Win 10: Development when working on Windows specific tasks (Excel, Word, etc)
The key point is that even though the WSL and Windows environments can “talk” to each other, there does need to be some segregation of responsibilities. For instance, when using git in WSL, it is recommended that you operate on the files in the WSL environment. The same goes for Windows - don’t try to run Windows executables directly from the WSL file system.
Finally, I still recommend that you use conda environments to keep your python environments clean. I have chosen to have a conda environment on Ubuntu and one on Windows so that I can make sure blog posts work appropriately across Windows and Linux environments.
Conclusion
WSL is a major step forward in making Windows a first class development platform. I have been a long-time Ubuntu user at home and Windows user at work. WSL finally gives me a platform where I can have the best of both worlds. I get access to all the tools and flexibility of working in Ubuntu alongside the common MS Office tools. In addition, I am confident any commercial software I might need can be installed on this system.
I hope you find this guide useful and that it helps you build your own python development environment for Windows and Linux. If you have any other tips, let me know in the comments.
from Planet Python
via read more
No comments:
Post a Comment