In a pythonic mind, we shoud keep things simple as possible.
But when having to deal with distribution of your code, especially if it's ONE small python file, you'll might cringe when facing the boilerplate that is required. Even with distribute in place of setuptools.
And speaking of packaging system, python has a very complex history of them (between distutils, setuptools, distribute, distutils2), adding a considerable burden to get a clear idea of what system you should use, and get a complete documentation of one system. Especially when whole part of each system share code, ideas, compatibility but aims to be better in some obscure part for a newcomer.
Notice how this stackoverflow question about python packaging systems gets answered brilliantly and how concrete differences between all these packaging solutions weren't mentioned.
However, I wanted a dead simple way to distribute my single file package. And I found 2 ways:
- one with distribute (the actually recommended packaging system),
- one with distutils2 (the upcoming-but-not-finished replacement of distribute).
I realized that I could illustrate some simple differences for newcomers when facing the need to package a simple python file.
The goal
You have a file foo.py holding whatever python code you want. foo.py is alone in a directory.
Install-ability
You need to be able to type python setup.py install (or something similar) in the source directory and all the magic required will fire out to set things how they are supposed to be in your python libraries, allowing you to invoke import foo in any of your python file without any unfriendly exception being cast.
distributability
Next, you should be able to type python setup.py sdist register upload (or something similar) to build a source distribution, register the package to the PyPI, and send it. So that, finally, from any computer having access to internet, you could cast a pip install foo (or something similar) to get your package downloaded from the PyPI, and installed.
Distribute way
This is the simplest way I found to distribute a ONE small python file using distribute, which seems to be the current best solution waiting for distutils2.
Requirements
Make sure you have the last distribute installed. You can install it with this command:
curl -O http://python-distribute.org/distribute_setup.py python distribute_setup.py
setup.py file
Then create a setup.py file in the same directory than foo.py, containing:
from setuptools import setup
setup(
## Required for ``python setup.py install``
name="foo_utils",
version="0.1",
py_modules=["foo"]
## Required for ``python setup.py sdist upload install``
)
The name will be your package name and version will be its version, these are used for naming the .egg and PyPI references.
The following is a less documented option and makes the solution I have found so small: py_modules is the python module you want to distribute. Typing foo targets the foo.py file.
Actually, complete reference of this trick can be found in distutils documentation, I couldn't find any reference to py_modules anywhere in the distribute documentation.
Tada !
The small setup.py you've written should allow you to type:
python setup.py install
To send it to PyPI, you'll need some other metadata as the author, summary, licence and maybe a small description. These are to be added in your setup.py file, and then:
python setup.py sdist register upload
... will send it to PyPI.
At last you'll then be able to install it from anywhere with:
pip install foo_utils
Distutils2 way
The major modification that you'll notice here is the fact that the configuration is not executable code anymore: setup.py is replaced by setup.cfg.
Requirements
You need to install distutils2:
pip install distutils2
If you have any trouble installing distutils2, you might be interested in the ending section about the truth about distutils2. You could also try something like that (adapt the version of course !):
cd /tmp wget https://pypi.python.org/packages/source/D/Distutils2/Distutils2-1.0a4.tar.gz tar xvzf Distutils2-1.0a4.tar.gz cd Distutils2-1.0a4 python setup.py install
setup.cfg
In the same directory than foo.py, you can create setup.cfg containing:
[metadata] name = foo_utils version = 0.1 [files] modules = foo
Notice that py_modules is in a files section and has been renamed to modules.
This first version of setup.cfg will allow you to achieve first goal of "install-ability". But you'll need more to get your code sent to PyPI:
[metadata]
name = foo_utils
version = 0.1.0
summary = A Simple example Foo program
description-file = README.rst CHANGELOG.rst
## sdist info
author = John Doe
author_email = john.doe@example.com
classifier =
Development Status :: 3 - Alpha
License :: OSI Approved :: GNU General Public License (GPL)
[files]
modules = foo_utils
extra_files =
README.rst
CHANGELOG.rst
Well, it's not that small, but I'm not sure what is actually really required, and I think a small summary or description won't hurt. Notice that we have to repeat our selves for the inclusion of the README.rst and CHANGELOG.rst files that are used in the description.
Of course, all details given hear are subject to changes in the upcoming releases of distutils2.
Tada !
The small setup.cfg you've written should allow you to type:
pysetup install
To reach the "install-ability" goal. And:
pysetup run sdist register upload
To send it to PyPI. Then you should be able to install it from anywhere with:
pysetup install foo_utils
Note
packages made with distutils2 won't be compatible with pip install PACKAGE commands as they do not provide any setup.py. You'll receive IOError: [Errno 2] No such file or directory.
If any of these commands don't work, jump ahead to the next section !.
The truth about distutils2
Before getting something actually working, I ran in multiple issues, and these are some of the trick I did to make it work.
First, get it installed:
cd /tmp wget https://pypi.python.org/packages/source/D/Distutils2/Distutils2-1.0a4.tar.gz tar xvzf Distutils2-1.0a4.tar.gz cd Distutils2-1.0a4 python setup.py install
Then, I proceeded towards the distribute installation:
$ pysetup foo_utils No handlers could be found for logger "distutils2"
Hum, after hacking into the distutils2 code, I've found that distutils hadn't any default logger, once one set, it was only to realize that it wanted to display a simple error to tell me that I messed with my arguments. The correct command is:
root@myhost$ pysetup install foo_utils Checking the installation location... Unable to write in "/usr/lib/python2.7/site-packages". Do you have the permissions ? root@myhost$
What are you talking about ?! I'm root ! After looking around, I found there were no such directory site-packages in my testing VM, which should mimick an Ubuntu 12.04 standard install. I pursued by:
root@myhost$ mkdir /usr/lib/python2.7/site-packages root@myhost$ pysetup install foo_utils Checking the installation location... Getting information about 'foo_utils'... u'python-debian': u'0.1.21ubuntu1' is not a valid version (field 'Version') u'python-apt': u'0.8.3ubuntu7' is not a valid version (field 'Version') u'ufw': u'0.31.1-1' is not a valid version (field 'Version') u'Landscape Client': u'12.05' is not a valid version (field 'Version') Installing u'foo_utils' 0.1.0... running install_dist running build running build_py running install_lib byte-compiling /usr/lib/python2.7/site-packages/foo.py to foo.pyc running install_distinfo creating /usr/lib/python2.7/site-packages/foo-utils-0.1.0.dist-info creating /usr/lib/python2.7/site-packages/foo-utils-0.1.0.dist-info/METADATA creating /usr/lib/python2.7/site-packages/foo-utils-0.1.0.dist-info/INSTALLER creating /usr/lib/python2.7/site-packages/foo-utils-0.1.0.dist-info/REQUESTED creating /usr/lib/python2.7/site-packages/foo-utils-0.1.0.dist-info/RECORD root@myhost$
Success ! I then did the test to see if I could "import" my lib:
root@myhost$ python Python 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import foo Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named foo >>>
Failure ! A quick check to sys.path will confirm that if the "site-package" was not existent, it wasn't included in the sys.path... which makes sense. I'm not aware of all these details, but why distutils2 then tries to install packages in this directory ?
Anyway, if I want to make sure that site-packages gets in sys.path, I can follow superuser advice:
echo ../site-packages > /usr/lib/python2.7/dist-packages/site-packages.pth
This will easily add site-packages directory system wide.
Conclusion
Is the packaging system of python dead-simple for one file ? Well, yes, once a straightforward recipe was discovered painfully.
Is it easy to find clear and unambiguous documentation ? Not really.
Is distutils2 finished ? Nope. Did you try with last available code from hg.python.org ? Yes.
Do you still need to use distribute ? Yes.
Any comments ?