Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions Doc/library/importlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ ABC hierarchy::
| +-- MetaPathFinder
| +-- PathEntryFinder
+-- Loader
+-- ResourceReader
+-- ResourceLoader --------+
+-- InspectLoader |
+-- ExecutionLoader --+
Expand Down Expand Up @@ -465,6 +466,72 @@ ABC hierarchy::
The import machinery now takes care of this automatically.


.. class:: ResourceReader

An :term:`abstract base class` for :term:`loaders <loader>`
representing a :term:`package` to provide the ability to read
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads a bit awkwardly for me. What about "An abstract base class for package loaders which provide the ability to read resources"?

*resources*.

From the perspective of this class, a *resource* is a binary
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it say "from the perspective of this ABC"?

artifact that is shipped within the package that this loader
represents. Typically this is something like a data file that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's another awkward turn of a similar phrase. "package that this loader represents" seems weird to me ;)

lives next to the ``__init__.py`` file of the package. The
purpose of this class is to help abstract out the accessing of
such data files so that it does not matter if the package and
its data file(s) are stored in a e.g. zip file versus on the
file system.

For any of methods of this class, a *resource* argument is
expected to be a :term:`file-like object` which represents
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So open_resource takes a file object and returns a file object? I may be misunderstanding here...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that term should say that a resource is a path-like object. I'll fix that in my follow up branch.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conceptually just a file name. This means that no subdirectory
paths should be included in the *resource* argument. This is
because the location of the package that the loader is for acts
as the "directory". Hence the metaphor for directories and file
names is packages and resources, respectively. This is also why
instances of this class are expected to directly correlate to
a specific package (instead of potentially representing multiple
packages or a module).

.. versionadded:: 3.7

.. abstractmethod:: open_resource(resource)

Returns an opened, :term:`file-like object` for binary reading
of the *resource*.

If the resource cannot be found, :exc:`FileNotFoundError` is
raised.

.. abstractmethod:: resource_path(resource)

Returns the file system path to the *resource*.

If the resource does not concretely exist on the file system,
raise :exc:`FileNotFoundError`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... is the same error returned when the given resource doesn't exist, or when the given resource exists but underlying storage is not a regular directory (e.g. a zip file)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. In either case, importlib.resources will know what to do with that (e.g. extract it to a temporary file in the case where it exists in the loader's view of the world, but not on the physical file system).


.. abstractmethod:: is_resource(path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the standalone package I called this name so that it doesn't necessarily imply a path (which could include slashes).


Returns ``True`` if the named *path* is considered a resource.
:exc:`FileNotFoundError` is raised if *path* does not exist.

.. abstractmethod:: contents()

Returns an :term:`iterator` of strings over the contents of
the package. Due note that it is not required that all names
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Due/Do/

returned by the iterator be actual resources, e.g. it is
acceptable to return names for which :meth:`is_resource` would
be false.

Allowing non-resource names to be returned is to allow for
situations where how a package and its resources are stored
are known a priori and the non-resource names would be useful.
For instance, returning subdirectory names is allowed so that
when it is known that the package and resources are stored on
the file system then those subdirectory names can be used.

The abstract method returns an empty iterator.


.. class:: ResourceLoader

An abstract base class for a :term:`loader` which implements the optional
Expand Down
38 changes: 38 additions & 0 deletions Lib/importlib/abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -340,3 +340,41 @@ def set_data(self, path, data):
"""

_register(SourceLoader, machinery.SourceFileLoader)


class ResourceReader(Loader):

"""Abstract base class for loaders to provide resource reading support."""

@abc.abstractmethod
def open_resource(self, resource):
"""Return an opened, file-like object for binary reading.

The 'resource' argument is expected to represent only a file name
and thus not contain any subdirectory components.

If the resource cannot be found, FileNotFoundError is raised.
"""
raise FileNotFoundError

@abc.abstractmethod
def resource_path(self, resource):
"""Return the file system path to the specified resource.

The 'resource' argument is expected to represent only a file name
and thus not contain any subdirectory components.

If the resource does not exist on the file system, raise
FileNotFoundError.
"""
raise FileNotFoundError

@abc.abstractmethod
def is_resource(self, path):
"""Return True if the named 'path' is consider a resource."""
raise FileNotFoundError

@abc.abstractmethod
def contents(self):
"""Return an iterator of strings over the contents of the package."""
return iter([])
39 changes: 39 additions & 0 deletions Lib/test/test_importlib/test_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,45 @@ def test_get_filename(self):
) = test_util.test_both(InspectLoaderDefaultsTests)


class ResourceReader:

def open_resource(self, *args, **kwargs):
return super().open_resource(*args, **kwargs)

def resource_path(self, *args, **kwargs):
return super().resource_path(*args, **kwargs)

def is_resource(self, *args, **kwargs):
return super().is_resource(*args, **kwargs)

def contents(self, *args, **kwargs):
return super().contents(*args, **kwargs)


class ResourceReaderDefaultsTests(ABCTestHarness):

SPLIT = make_abc_subclasses(ResourceReader)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is SPLIT required by ABCTestHarness? It seems an odd choice of terms.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and I didn't name it.


def test_open_resource(self):
with self.assertRaises(FileNotFoundError):
self.ins.open_resource('dummy_file')

def test_resource_path(self):
with self.assertRaises(FileNotFoundError):
self.ins.resource_path('dummy_file')

def test_is_resource(self):
with self.assertRaises(FileNotFoundError):
self.ins.is_resource('dummy_file')

def test_contents(self):
self.assertEqual([], list(self.ins.contents()))

(Frozen_RRDefaultTests,
Source_RRDefaultsTests
) = test_util.test_both(ResourceReaderDefaultsTests)


##### MetaPathFinder concrete methods ##########################################
class MetaPathFinderFindModuleTests:

Expand Down