Python: Importing Packages

Importing and working with modules is a little different in Python than some other languages you may be familiar with, and since we are using it for some 3DChat technology—and hopefully in the future by using Django for more web applications—I would like to explain some of those differences.

You import a module in Python by using the appropriately named ‘import’ keyword. Let’s say we have a project with a directory structure like this:

/FooProject
|
|--- Application.py
|--- Connections.py
|--- /Parsers
*        |
*        |--- __init__.py
*        |--- JSON.py
*        |--- XML.py
*
|--- Reader.py

Our main code is in Application.py. It is what we run to execute the whole thing. I don’t know what the application does, but apparently it does some parsing of JSON and XML. So likely some of the other files would import those parsers. Let’s say in Reader.py we needed to import two classes: JSONParser and XMLParser. Here are some ways we can do that.

import Parsers

When Python sees an import statement it begins by searching each value of the ‘sys.path’ list. By default that list starts with the current directory. So in this case Python would see the ‘Parsers’ directory and import that. But what does that entail?

First note the strangely named __init__.py file, with two underscores on each end. Python requires this file to be present in order for a directory to be treated as a package ; that way directories with common names cannot unintentionally conflict with standard library packages. The file is executed the /first/ time the package is imported in the application. It is often used to run start-up code to prepare a package for use. It is also very often empty. It does not
need anything in it—it just needs to be present.

Since Python sees that file it knows the directory represents a package. It then takes all of the modules in that package directory and makes them available in the current namespace in the importing module (Reader.py). However, you still have to refer to the imported modules with fully-qualified names! This often trips up people. That means in Reader.py we have to write this:

import Parsers
xml_parser = Parsers.XML.XMLParser()

The full name is ‘Package.Module.Class’. It is not accidentally that this maps directly to ‘Directory.Filename.Class’.

If we just wrote XMLParser() we would get an error, because this form of importing still requires us to use the full name for accessing the classes. This helps prevent conflicts from using different packages that provide modules with the same names.

import Parsers
import OtherParsers
foo = Parsers.XML.XMLParser()
bar = OtherParsers.XML.XMLParser()

No problems here. We can use both XML.XMLParser classes just fine.
This is a safe way of importing a whole package.

import Parsers.JSON

If we only need one module from the package then we can write this. In our code we still have to refer to the class by the full name ‘Parsers.JSON.JSONParser’, but with the above we would not have access to the XMLParser. This is useful when we only need a few parts of a package; we can specify clearly in our code what we are using.

from Parsers.JSON import JSONParser

This is an example of the other importing syntax in Python: the from ... import ... construct. This has a similar effect to the previous import, in that it only gives us access to the JSONParser class.

However, the key difference is that this syntax brings the imported class into our current symbol table. So no longer would we have to us the ‘Parsers’ prefix. We could write code like this:

from Parsers import JSON
parser = JSON.JSONParser()
Or we could write it this way:
from Parsers.JSON import JSONParser
parser = JSONParser()

This can be more convenient than typing out the full name, especially when you are using deeply-nested packages. But it opens us up to potential naming conflicts.

from MyParsers import XMLParser
from YourParsers import XMLParser

Now we have an error, because the current module would not know which XMLParser is which.

from Parsers import *

Finally we have this form, which would import both modules in our example—maybe. I mention this one last because you should stay away from it. I didn’t want to not teach it to you, because you will likely see it here and there in some applications, but the Python community frowns upon this practice. Unless you really know the package you are importing, you are opening yourself up to nasty conflicts in the symbol table.

As I said, writing ‘from Parsers import *’ may import all of the
modules. When Python sees that syntax it looks inside of the
__init__.py file for a list called __all__. That list contains the names of the modules to export for ‘import *’. In our example, we could put this inside of the init file

__all__ = (‘JSON’, ‘XML’)

If __all__ is not defined then ‘from Parsers import *’ will not automatically import all sub-packages. It will only ensure that Parsers itself is imported, by running __init__.py.

That is another reason to avoid this form.

from . import XMLParser

I said finally on the last one, but there’s one twist I need to mention. The ‘from’ syntax lets us import using a relative path. For example, we could write the above inside of JSON.py if we needed to use the XMLParser class inside there. You can write ‘from ..’ to pull something from the parent package as well.

And what if an import fails? Python will throw an ImportError exception, which we can check for. This can be really useful if we are trying to import something but want to have a contingency plan in place.

try:
import Parsers
except ImportError:
import BackupParsers as Parsers

Hey—new syntax, ‘import as’. I think the meaning is obvious, yes? You can try to import something else as a substitute. Or if that fails you can try to make it up on the spot.

try:
from Parsers.XML import XMLParser
except ImportError:
class XMLParser(object):

…

So next time you need to import something in Python, keep these things in mind. You should feel encouraged to use the ‘from … import’ syntax to save yourself typing, but leave at least one layer of the package namespace intact so that you avoid conflicts.

0 Comment | Posted in Python,blog by admin on January 20, 2010