OpenERP 模块动态加载原理及启动代码分析
-
一般来说我们在编程中，对象定义都是预先定义好的。一些 OOP 语言（包括 Python/Java）允许对象是 自省的（也称为 反射）。即，自省对象能够描述自己：实例属于哪个类？类有哪些祖先？对象可以用哪些方法和属性？自省让处理对象的函数或方法根据传递给函数或方法的对象类型来做决定。即允许对象在运行时动态改变方法成员等属性。
得益于OpenERP ORM 模型的精巧设计，实际上 OpenERP 运行时也是动态读取模块信息并动态构建对象的。如在模块开发中，继承了 'res.users', 新增一个方法或新增一个字段。在OpenERP 导入该模块后， OpenERP 会马上重构 'res.users' 对象并将新增的方法或字段添加到该对象。那么，OpenERP 是如何做到这点的呢？ 让我们从OpenERP 的启动部分开始分析：
首先，OpenERP 启动相关的服务, 这时并没有建立数据库链接和载入对象<br /> if not config["stop_after_init"]:<br /> setup_pid_file()<br /> # Some module register themselves when they are loaded so we need the<br /> # services to be running before loading any registry.<br /> if config['workers']:<br /> openerp.service.start_services_workers()<br /> else:<br /> openerp.service.start_services()<br />
不过可以在配置文件中指定 'db_name' 参数，可以让 OpenERP 在启动时加载指定数据库的对象并启动 Cron。 实际生产环境中建议启用该参数，否则需要在启动OpenERP后，登录一次OpenERP 在会加载对象并启动CRON<br /> if config['db_name']:<br /> for dbname in config['db_name'].split(','):<br /> preload_registry(dbname)<br />
不指定 'db_name' 参数情况下，OpenERP 会直到用户登录时才会初始化指定数据库。<br />def login(db, login, password):<br /> pool = pooler.get_pool(db)<br /> user_obj = pool.get('res.users')<br /> return user_obj.login(db, login, password)<br />
打开 get_db 和 get_db_and_pool 定义<br />def get_db(db_name):<br /> """Return a database connection. The corresponding registry is initialized."""<br /> return get_db_and_pool(db_name)[0]<br /><br /><br />def get_db_and_pool(db_name, force_demo=False, status=None, update_module=False):<br /> """Create and return a database connection and a newly initialized registry."""<br /> registry = RegistryManager.get(db_name, force_demo, status, update_module)<br /> return registry.db, registry<br /><br />
顺藤摸瓜，看RegistryManager.new 定义<br /> try:<br /> # This should be a method on Registry<br /> openerp.modules.load_modules(registry.db, force_demo, status, update_module)<br /> except Exception:<br /> del cls.registries[db_name]<br /> raise<br />
然后看到 load_modules, 终于要加载了，嘿嘿。<br /><br /> # STEP 1: LOAD BASE (must be done before module dependencies can be computed for later steps) <br /> graph = openerp.modules.graph.Graph()<br /> graph.add_module(cr, 'base', force)<br /> if not graph:<br /> _logger.critical('module base cannot be loaded! (hint: verify addons-path)')<br /> raise osv.osv.except_osv(_('Could not load base module'), _('module base cannot be loaded! (hint: verify addons-path)'))<br /><br /> # processed_modules: for cleanup step after install<br /> # loaded_modules: to avoid double loading<br /> report = pool._assertion_report<br /> loaded_modules, processed_modules = load_module_graph(cr, graph, status, perform_checks=update_module, report=report)<br /><br /> if tools.config['load_language']:<br /> for lang in tools.config['load_language'].split(','):<br /> tools.load_language(cr, lang)<br /><br /> # STEP 2: Mark other modules to be loaded/updated<br /> if update_module:<br /> modobj = pool.get('ir.module.module')<br /> if ('base' in tools.config['init']) or ('base' in tools.config['update']):<br /> _logger.info('updating modules list')<br /> modobj.update_list(cr, SUPERUSER_ID)<br /><br /> _check_module_names(cr, itertools.chain(tools.config['init'].keys(), tools.config['update'].keys()))<br /><br /> mods = [k for k in tools.config['init'] if tools.config['init'][k]]<br /> if mods:<br /> ids = modobj.search(cr, SUPERUSER_ID, ['&', ('state', '=', 'uninstalled'), ('name', 'in', mods)])<br /> if ids:<br /> modobj.button_install(cr, SUPERUSER_ID, ids)<br /><br /> mods = [k for k in tools.config['update'] if tools.config['update'][k]]<br /> if mods:<br /> ids = modobj.search(cr, SUPERUSER_ID, ['&', ('state', '=', 'installed'), ('name', 'in', mods)])<br /> if ids:<br /> modobj.button_upgrade(cr, SUPERUSER_ID, ids)<br /><br /> cr.execute("update ir_module_module set state=%s where name=%s", ('installed', 'base'))<br /><br /><br /> # STEP 3: Load marked modules (skipping base which was done in STEP 1)<br /> # IMPORTANT: this is done in two parts, first loading all installed or<br /> # partially installed modules (i.e. installed/to upgrade), to<br /> # offer a consistent system to the second part: installing<br /> # newly selected modules.<br /> # We include the modules 'to remove' in the first step, because<br /> # they are part of the "currently installed" modules. They will<br /> # be dropped in STEP 6 later, before restarting the loading<br /> # process.<br /> states_to_load = ['installed', 'to upgrade', 'to remove']<br /> processed = load_marked_modules(cr, graph, states_to_load, force, status, report, loaded_modules, update_module)<br /> processed_modules.extend(processed)<br /> if update_module:<br /> states_to_load = ['to install']<br /> processed = load_marked_modules(cr, graph, states_to_load, force, status, report, loaded_modules, update_module)<br /> processed_modules.extend(processed)<br /><br />
这里，第一步是加载核心模块 ['base']，第二步加载需要升级或预载的模块，第三步加载已安装的模块。实际加载语句是：<br /> processed = load_marked_modules(cr, graph, states_to_load, force, status, report, loaded_modules, update_module)<br />
查看 load_marked_module 定义：<br />def load_marked_modules(cr, graph, states, force, progressdict, report, loaded_modules, perform_checks):<br /> """Loads modules marked with ``states``, adding them to ``graph`` and<br /> ``loaded_modules`` and returns a list of installed/upgraded modules."""<br /> processed_modules = []<br /> while True:<br /> cr.execute("SELECT name from ir_module_module WHERE state IN %s" ,(tuple(states),))<br /> module_list = [name for (name,) in cr.fetchall() if name not in graph]<br /> graph.add_modules(cr, module_list, force)<br /> _logger.debug('Updating graph with %d more modules', len(module_list))<br /> loaded, processed = load_module_graph(cr, graph, progressdict, report=report, skip_modules=loaded_modules, perform_checks=perform_checks)<br /> processed_modules.extend(processed)<br /> loaded_modules.extend(loaded)<br /> if not processed: break<br /> return processed_modules<br />
重点是 load_module_graph<br /><br /> # register, instantiate and initialize models for each modules<br /> for index, package in enumerate(graph):<br /> module_name = package.name<br /> module_id = package.id<br /><br /> if skip_modules and module_name in skip_modules:<br /> continue<br /><br /> _logger.debug('module %s: loading objects', package.name)<br /> migrations.migrate_module(package, 'pre')<br /> load_openerp_module(package.name)<br /><br /> models = pool.load(cr, package)<br /><br /> loaded_modules.append(package.name)<br /> if hasattr(package, 'init') or hasattr(package, 'update') or package.state in ('to install', 'to upgrade'):<br /> init_module_models(cr, package.name, models)<br /> pool._init_modules.add(package.name)<br /> status['progress'] = float(index) / len(graph)<br /><br />
主要是下面2句，<br /> load_openerp_module(package.name)<br /><br /> models = pool.load(cr, package)<br /><br />
先看 load_openerp_module<br /> initialize_sys_path()<br /> try:<br /> mod_path = get_module_path(module_name)<br /> zip_mod_path = '' if not mod_path else mod_path + '.zip'<br /> if not os.path.isfile(zip_mod_path):<br /> __import__('openerp.addons.' + module_name)<br /> else:<br /> zimp = zipimport.zipimporter(zip_mod_path)<br /> zimp.load_module(module_name)<br />
上面代码中 import 了一个模块。如果您看过 digitalsatori 校长的大作 OpenERP与Python 元编程, 下面就涉及到元类了：
import 的时候 就会调用元类的构造函数<br /> def __init__(self, name, bases, attrs):<br /> if not self._register:<br /> self._register = True<br /> super(MetaModel, self).__init__(name, bases, attrs)<br /> return<br /><br /> # The (OpenERP) module name can be in the `openerp.addons` namespace<br /> # or not. For instance module `sale` can be imported as<br /> # `openerp.addons.sale` (the good way) or `sale` (for backward<br /> # compatibility).<br /> module_parts = self.__module__.split('.')<br /> if len(module_parts) > 2 and module_parts[0] == 'openerp' and \<br /> module_parts[1] == 'addons':<br /> module_name = self.__module__.split('.')[2]<br /> else:<br /> module_name = self.__module__.split('.')[0]<br /> if not hasattr(self, '_module'):<br /> self._module = module_name<br /><br /> # Remember which models to instanciate for this module.<br /> if not self._custom:<br /> self.module_to_models.setdefault(self._module, []).append(self)<br /><br />
上面的代码基本上就是将自身类加入到 module_to_models 字典中。
然后我们来看pool.load<br /> def load(self, cr, module):<br /> """ Load a given module in the registry.<br /><br /> At the Python level, the modules are already loaded, but not yet on a<br /> per-registry level. This method populates a registry with the given<br /> modules, i.e. it instanciates all the classes of a the given module<br /> and registers them in the registry.<br /><br /> """<br /> models_to_load = [] # need to preserve loading order<br /> # Instantiate registered classes (via the MetaModel automatic discovery<br /> # or via explicit constructor call), and add them to the pool.<br /> for cls in openerp.osv.orm.MetaModel.module_to_models.get(module.name, []):<br /> # models register themselves in self.models<br /> model = cls.create_instance(self, cr)<br /> if model._name not in models_to_load:<br /> # avoid double-loading models whose declaration is split<br /> models_to_load.append(model._name)<br /> return [self.models[m] for m in models_to_load]<br /><br />
这里我们可以看到 MetaModel 的身影，cls.create_instance(self, cr) 这里就是动态构造对象的核心代码。<br /><br /> parent_names = getattr(cls, '_inherit', None)<br /> # 判断是否有继承父类 <br /> if parent_names:<br /> if isinstance(parent_names, (str, unicode)):<br /> name = cls._name or parent_names<br /> parent_names = [parent_names]<br /> else:<br /> name = cls._name<br /> if not name:<br /> raise TypeError('_name is mandatory in case of multiple inheritance')<br /><br /> for parent_name in ((type(parent_names)==list) and parent_names or [parent_names]):<br /> # 读取父类<br /> parent_model = pool.get(parent_name)<br /> if not parent_model:<br /> raise TypeError('The model "%s" specifies an unexisting parent class "%s"\n'<br /> 'You may need to add a dependency on the parent class\' module.' % (name, parent_name))<br /> if not getattr(cls, '_original_module', None) and name == parent_model._name:<br /> cls._original_module = parent_model._original_module<br /> parent_class = parent_model.__class__<br /> nattr = {}<br /> # 复制父类属性<br /> for s in attributes:<br /> new = copy.copy(getattr(parent_model, s, {}))<br /> if s == '_columns':<br /> # Don't _inherit custom fields.<br /> for c in new.keys():<br /> if new[c].manual:<br /> del new[c]<br /> # Duplicate float fields because they have a .digits<br /> # cache (which must be per-registry, not server-wide).<br /> for c in new.keys():<br /> if new[c]._type == 'float':<br /> new[c] = copy.copy(new[c])<br /> if hasattr(new, 'update'):<br /> new.update(cls.__dict__.get(s, {}))<br /> elif s=='_constraints':<br /> for c in cls.__dict__.get(s, []):<br /> exist = False<br /> for c2 in range(len(new)):<br /> #For _constraints, we should check field and methods as well<br /> if new[c2][2]==c[2] and (new[c2][0] == c[0] \<br /> or getattr(new[c2][0],'__name__', True) == \<br /> getattr(c[0],'__name__', False)):<br /> # If new class defines a constraint with<br /> # same function name, we let it override<br /> # the old one.<br /><br /> new[c2] = c<br /> exist = True<br /> break<br /> if not exist:<br /> new.append(c)<br /> else:<br /> new.extend(cls.__dict__.get(s, []))<br /> nattr[s] = new<br /><br /> # Keep links to non-inherited constraints, e.g. useful when exporting translations<br /> nattr['_local_constraints'] = cls.__dict__.get('_constraints', [])<br /> nattr['_local_sql_constraints'] = cls.__dict__.get('_sql_constraints', [])<br /> # 调用元类构造函数<br /> cls = type(name, (cls, parent_class), dict(nattr, _register=False))<br /> else:<br /> cls._local_constraints = getattr(cls, '_constraints', [])<br /> cls._local_sql_constraints = getattr(cls, '_sql_constraints', [])<br /><br /> if not getattr(cls, '_original_module', None):<br /> cls._original_module = cls._module<br /> # 构造对象<br /> obj = object.__new__(cls)<br /> # 初始化对象<br /> obj.__init__(pool, cr)<br /> return obj<br />
上面代码很重要，可以看到首先是判断该对象是否有继承父类，如果没有就直接构造，和动态没有什么关系。
如果有继承父类, 就复制父类属性, 这里就是动态构建类的做法。
假如有不同模块，都继承了同一个父类，那么如何保证类成员和属性是否加载完整或覆盖呢？ 答案在于这句代码：<br /> parent_model = pool.get(parent_name)<br />
Registry.get 的定义<br /> def get(self, model_name):<br /> """ Return a model for a given name or None if it doesn't exist."""<br /> return self.models.get(model_name)<br />
最后看看obj.init(pool, cr)初始化对象，做了什么动作？<br /> def __init__(self, pool, cr):<br /> """ Initialize a model and make it part of the given registry.<br /><br /> - copy the stored fields' functions in the osv_pool,<br /> - update the _columns with the fields found in ir_model_fields,<br /> - ensure there is a many2one for each _inherits'd parent,<br /> - update the children's _columns,<br /> - give a chance to each field to initialize itself.<br /><br /> """<br /> pool.add(self._name, self)<br /> self.pool = pool<br />
pool.add(self._name, self) 定义如下：<br /> def add(self, model_name, model):<br /> """ Add or replace a model in the registry."""<br /> self.models[model_name] = model<br />
到这里应该很非常清楚，Registry.models 保存了对象的 model 信息。这样多个对象继承同一父类时，按照加载顺序先后动态构建相关的类。
至此，OpenERP 启动时动态加载模块分析完成。如模块安装、升级、卸载等, 则是通过 signal_registry_change 和 check_registry_signaling 处理，重新载入 Registry, 然后重新构建 OpenERP 对象。
转自 http://buke.github.com/blog/2013/02/26/openerp-dynamic-loading-and-booting-way/ br /> -
mark
