[modutils] Use a cache when abspathing sys.path for file_from_modpath.

This adds a dumb cache. There's no eviction, so there's the risk of eating memory if sys.path is changed frequently or _module_file is called very many times with different path arguments, but those both seem unlikely.

Closes #235354.

authorEevee (Alex Munroe) <amunroe@yelp.com>
changesete0d972362b72
branchdefault
phasedraft
hiddenyes
parent revision#3ff9bfb473eb [graph] More explicit error message if dot is not found/executable (closes #253516)
child revision#05114ba5f0d7 [configuration] extract a YAML-friendly build_config_struct function from generate_config (closes #204933)
files modified by this revision
modutils.py
# HG changeset patch
# User Eevee (Alex Munroe) <amunroe@yelp.com>
# Date 1396982519 25200
# Tue Apr 08 11:41:59 2014 -0700
# Node ID e0d972362b721b0cb719c7e443afd198ba30ac8d
# Parent 3ff9bfb473eb2164b07106d0e72e76bd38f33a64
[modutils] Use a cache when abspathing sys.path for file_from_modpath.

This adds a dumb cache. There's no eviction, so there's the risk of eating
memory if sys.path is changed frequently or _module_file is called very many
times with different path arguments, but those both seem unlikely.

Closes #235354.

diff --git a/modutils.py b/modutils.py
@@ -581,10 +581,20 @@
1  try:
2      import pkg_resources
3  except ImportError:
4      pkg_resources = None
5 
6 +_abspathcache = {}
7 +def _abspath(path):
8 +    """abspath with caching"""
9 +    # _module_file calls abspath on every path in sys.path every time it's
10 +    # called; on a larger codebase this easily adds up to half a second just
11 +    # assembling path components.  this cache alleviates that.
12 +    if path not in _abspathcache:
13 +        _abspathcache[path] = abspath(path)
14 +    return _abspathcache[path]
15 +
16  def _module_file(modpath, path=None):
17      """get a module type / file path
18 
19      :type modpath: list or tuple
20      :param modpath:
@@ -637,11 +647,11 @@
21              if checkeggs:
22                  return _search_zip(modpath, pic)[:2]
23              raise
24          else:
25              if checkeggs and mp_filename:
26 -                fullabspath = [abspath(x) for x in _path]
27 +                fullabspath = [_abspath(x) for x in _path]
28                  try:
29                      pathindex = fullabspath.index(dirname(abspath(mp_filename)))
30                      emtype, emp_filename, zippath = _search_zip(modpath, pic)
31                      if pathindex > _path.index(zippath):
32                          # an egg takes priority