33Creating an extension
44=====================
55
6- Once an new filesystem implemented, it is possible to distribute as a
7- subpackage contained in the ``fs `` namespace. Let's say you are trying
8- to create an extension for a filesystem called **AwesomeFS **.
6+ Once a filesystem has been implemented, it can be integrated with other
7+ projects and applications using PyFilesystem.
98
109
1110Naming Convention
1211-----------------
1312
14- For the sake of clarity, and to give a clearer sight of the
15- Pyfilesystem2 ecosystem, your extension should be called **fs.awesome **
16- or **fs.awesomefs **, since PyPI allows packages to be namespaced. Let us
17- stick with **fs.awesome ** for now.
13+ For visibility in PyPi, we recommend that your package be prefixed with
14+ ``fs- ``, for example your package may be named `fs-awesome ` or `fs-
15+ awesomefs `.
1816
1917
20- Extension Structure
21- -------------------
18+ Opener
19+ ------
2220
23- The extension must have either of the following structures: ::
21+ In order for your filesystem to be opened through :doc: `openers ` like
22+ the other builtin filesystems, you should define a
23+ :class: `~fs.opener.base.Opener ` class for your filesystem.
2424
25- └── fs.awesome └── fs.awesome
26- ├── fs ├── fs
27- │ ├── awesomefs.py │ ├── awesomefs
28- │ └── opener | | ├── __init__.py
29- │ └── awesomefs.py | | ├── some_file.py
30- └── setup.py | | └── some_other_file.py
31- │ └── opener
32- │ └── awesomefs.py
33- └── setup.py
25+ Here's an example taken from an Amazon S3 Filesystem::
3426
3527
36- The structure on the left will work fine if you only need a single file
37- to implement **AwesomeFS **, but if you end up creating more,
38- you should probably use the structure on the right (create a package
39- instead of a single file).
28+ """Defines the S3FSOpener."""
4029
41- .. warning ::
30+ __all__ = ['S3FSOpener']
4231
43- Do **NOT** create ``fs/__init__.py`` or ``fs/opener/__init__.py`` ! Since
44- those files are vital to the main Pyfilesystem2 package, including them
45- could result in having your extension break the whole Pyfilesystem2
46- package when installing.
32+ from fs.opener import Opener, OpenerError
4733
34+ from ._s3fs import S3FS
4835
4936
50-
51- ------
37+ class S3FSOpener(Opener):
38+ protocols = ['s3']
39+
40+ def open_fs(self, fs_url, parse_result, writeable, create, cwd):
41+ bucket_name, _, dir_path = parse_result.resource.partition('/')
42+ if not bucket_name:
43+ raise OpenerError(
44+ "invalid bucket name in '{}'".format(fs_url)
45+ )
46+ s3fs = S3FS(
47+ bucket_name,
48+ dir_path=dir_path or '/',
49+ aws_access_key_id=parse_result.username or None,
50+ aws_secret_access_key=parse_result.password or None,
51+ )
52+ return s3fs
5253
53- In order for your filesystem to be opened through :doc: `openers ` like the
54- other builtin filesystems, you must declare an :class: `~fs.opener.base.Opener `.
55- For good practice, the implementation should be done in a file inside the
56- ``fs.opener `` module, so in our case, ``fs/opener/awesomefs.py ``. Let us call
57- the opener ``AwesomeFSOpener ``. Once done with the implementation, you must
58- declare the opener as an entry point in the setup file for ``fs.open_fs `` to
59- actually register this new protocol. See below for an example, or read about
60- `entry points <http://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=entry%20points#dynamic-discovery-of-services-and-plugins >`_
61- if you want to know more.
54+ My convention this would be declared in ``opener.py ``.
6255
6356
57+ To register the opener you will need to define an `entry point
58+ <http://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=entry%20points#dynamic-discovery-of-services-and-plugins> `_
59+ in your setup.py. See below for an example.
60+
6461
6562The ``setup.py `` file
6663---------------------
@@ -69,49 +66,40 @@ Refer to the `setuptools documentation <https://setuptools.readthedocs.io/>`_
6966to see how to write a ``setup.py `` file. There are only a few things that
7067should be kept in mind when creating a Pyfilesystem2 extension. Make sure that:
7168
72- * the name of the package is the *namespaced * name (**fs.awesome ** with our
73- example).
74- * ``fs ``, ``fs.opener `` and ``fs.awesomefs `` packages are included. Since
75- you can't create ``fs/__init__.py `` and ``fs/opener/__init__.py ``, setuptools
76- won't be able to find your packages if you use ``setuptools.find_packages ``,
77- so you will have to include packages manually.
78- * ``fs `` is in the ``install_requires `` list, in order to
79- always have Pyfilesystem2 installed before your extension.
80- * Ìf you created an opener, include it as an ``fs.opener `` entry point, using
81- the name of the entry point as the protocol to be used. If the protocol to
82- open ``AwesomeFS `` were ``awe:// ``, the entry point declaration would be::
83-
84- awe = fs.opener.awesomefs:AwesomeFS
85-
86- (format, like any other setuptools entry point, is ``protocol = module.submodule:OpenerClass ``).
87-
69+ * ``fs `` is in the ``install_requires `` list, in order to always have
70+ Pyfilesystem2 installed before your extension. You should reference
71+ the version number with the ``~= `` operator which ensures that the
72+ install will get any bugfix releases of PyFilesystem but not any
73+ potentially breaking changes.
74+ * Ìf you created an opener, include it as an ``fs.opener `` entry point,
75+ using the name of the entry point as the protocol to be used.
8876
8977Here is an minimal ``setup.py `` for our project:
9078
9179.. code :: python
9280
9381 from setuptools import setup
9482 setup(
83+ name = ' fs-awesomefs' # Name in PyPi
9584 author = " You !"
9685 author_email = " your.email@domain.ext"
9786 description = " An awesome filesystem for pyfilesystem2 !"
98- install_requires = [" fs"
99- entry_points = {' fs.opener'
100- ' awe = fs.opener.awesomefs:AwesomeFS'
101- ]},
87+ install_requires = [
88+ " fs~=2.0.5"
89+ ],
90+ entry_points = {
91+ ' fs.opener'
92+ ' awe = awesomefs.opener:AwesomeFSOpener'
93+ ]
94+ },
10295 license = " MY LICENSE"
103- name = ' fs.awesome'
104- packages = [' fs' ' fs.opener' ' fs.awesome' # if fs.awesomefs is a package
105- # packages=['fs', 'fs.opener'] # if fs.awesomefs is not a package
96+ packages = [' awesomefs'
10697 version = " X.Y.Z"
10798 )
10899
109100
110101--------------
111102
112- * Use `relative imports <https://www.python.org/dev/peps/pep-0328/#guido-s-decision >`_
113- whenever you try to access to a resource in the ``fs `` module or any of its
114- submodules.
115103* Keep track of your achievements ! Add ``__version__ ``, ``__author__ ``,
116104 ``__author_email__ `` and ``__license__ `` variables to your project
117105 (either in ``fs/awesomefs.py `` or ``fs/awesomefs/__init__.py `` depending
0 commit comments