Building native Python 3.4 module with Pip on Windows

Hello,

Yesterday I decided to use Python setproctitle module in a project to rename Python script process name (for pretty displaying in nestat, ps…).
RPM package for CentOS 7 was done very quickly by modifying current dev Fedora one to get Python34 flavor for my old CentOS but I try to keep compatibility of my code for Windows too (mostly for development purpose of colleagues).

As usual, I would just go for “pip install setproctitle” on Windows (I would clearly advice to NEVER do that on production but it’s fine for developing).
Sadly it failed with the following error:
error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).

According to Google this error is quite famous but most of the people seem to be trying to fix it without having any clue of what is really going on.
The root of this issue is that renaming a process in something really specific and thus platform dependent. If you look at setproctitle code you’ll see it’s all C code with specific section for each family of operating system. So we are having to issue installing this module on Windows because:

  • You need a compiler, but unlike on Linux you need the same compiler that the one Python team used when building the Python interpreter you have installed
  • You will probably also need Windows SDK, because setproctitle is very likely to use Windows low-level headers

According to pip error message when installing setproctitle module, I need Visual Studio 10.0 compiler. Okay.
Thanks to Wikipedia, I’m now aware that version 10.0 is actually Visual Studio 2010.
Microsoft confirms this but adds an interesting information: Visual Studio 2010 is a commercial software, so I need a free alternative which is Microsoft Windows SDK for Windows 7 and .NET Framework 4 embedding Visual Studio 2010 compiler.

I’d suggest getting the ISO version instead because the previous link in an online installer. It may not work anymore next time you need to install it…

A funny thing: you’ll be prompted for three different ISO file without any information about what the difference is… So here is the explanation:

  • GRMSDK_EN_DVD.iso: This is the regular X86 Windows running in 32 bits mode
  • GRMSDKIAI_EN_DVD.iso: Intel Itanium 64 bits, you don’t want that
  • GRMSDKX_EN_DVD.iso: X64 version, that’s probably the one you need

If you get the wrong one, the installer will fail with weird error message saying there’s an MSI file missing!

Before trying to install this, uninstall any “Visual Studio 2010” related software, especially the classic Microsoft Visual Studio 2010 Redistribuable X86 and X64 which are very likely to be installed already. Otherwise, the SDK will fail to install without any understandable error message (but you’re free to give it a try and try to figure out what’s going on in the Windows Installer log file, good luck)

I also had trouble running the installer from a network mapped drive, so you can safely extract ISO content with 7-Zip but you might have to copy the folder locally before running it (any feedback would be appreciated in comments if you give a try).

You may now think you’re ready but wait… There’s more.
It seems Windows SDK package installs a broken Visual Studio distribution: KB2519277 (FIX: Visual C++ compilers are removed when you upgrade Visual Studio 2010 […] if Windows SDK v7.1 is installed). According to the title, it’s not exactly what we are doing but you really need that, get VC-Compiler-KB2519277.exe here:

Last but no least, despite Microsoft released a fix to repair a broken Visual Studio installation from the SDK package, they still managed to release it broken: it’s not working on X64, there’s a missing BAT file to set environment variables when running from an X64 shell 😀 No kidding…

Even worse, it has been reported to Microsoft but they closed the issue with no explanation: https://connect.microsoft.com/VisualStudio/feedback/details/510784/vcvarsall-bat-amd64-environment-is-missing

Hopefully some people at StackOverflow fixed the issue by themselves.

I made a batch script so anyone can just run the script and enjoy the fix:

Again, it cannot be run from a network drive (wtf, Windows really…) so you’ll have to create this script on your desktop with “.bat” extension and run it with administrator privileges using right click.

Now you can go back to pip and enjoy the package building and installing successfully 🙂

PS: Did I mention setproctitle 1.1.10 module is real good shit ? If you’re running tones of Python processes, especially related to networking you may benefit from a renamed process when using ps or netstat !