AttributeError: module '__main__' has no attribute '_module_fqn' #849
Comments
anxstj
added
affects-0.3
|
I'm trying to look into this. The issue seems to be caused by the introduction of the module_respawn API in ansible in commit 4c5ce5a1a9e79a845aff4978cfeb72a0d4ecf7d6, an in particular by the change on line 197 of runpy.run_module(mod_name='%(module_fqn)s', init_globals=None, run_name='__main__', alter_sys=True)to runpy.run_module(mod_name='%(module_fqn)s', init_globals=dict(_module_fqn='%(module_fqn)s', _modlib_path=modlib_path),
run_name='__main__', alter_sys=True)In particular, it seems that mitogen_ansible doesn't insert My guess is that that attribute should be inserted into the target module somewhere around here https://github.com/mitogen-hq/mitogen/blob/master/ansible_mitogen/runner.py#L970 but simply adding mod._module_fqn = self.py_module_namethere has no effect. I'm lacking knowledge of python internals in order to get mitogen to insert this |
|
can confirm the lastest master still has this problem |
|
Happens on a Fedora 35 controller node w/ OracleLinux 8.5 managed nodes using the |
I put it here. Seems to have addressed the issue. mitogen/ansible_mitogen/runner.py Lines 526 to 528 in a564d8a |
|
@phemmer What exactly have you inserted there? This code doesn't even seem to be called when running a module such as apt (at least, putting a generic exception there had no effect whatsoever). |
|
I think this issue is triggered when the ansible_python_interpreter being used on the target is not the system python. The ansible modules that deal with packages require python libraries that are not installable with pip, and can't reasonable be streamed out by mitogen because of that. In the ansible module code, there's a breakout were it will respawn into the system python in order to find these libraries: https://github.com/ansible/ansible/blob/6e57c8c0844c44a102dc081b93d5299cae9619e2/lib/ansible/module_utils/common/respawn.py#L18 |
|
@mhenniges I think the situation you describe is correct, but it still should be supported by mitogen, I think. I don't think it is hard to solve (you need to insert two globals in the target module's main namespace), I only don't know enough about the mitogen internals to determine how/where to do that. |
|
@mhenniges Explicitly setting Setting it to I'm testing this with molecule and it sets |
|
I have Ansible uses As already mentioned, for me, the error only occurs in the |
|
I think you are missing the point of this bug; sure, I'd be nice to have a workaround by setting the python interpreter to a different binary, but that still wouldn't fix this bug. The problem is that the ansible API has changed, and this breaks specific mitogen workflows. The solution should thus be to fix mitogen, not to avoid the specific circumstances in which this workflow is triggered. |
|
I'm not sure what has happened, but eveything now Just Work with ansible-core 2.12.2 and mitogen 0.3.2. |
Worked for few runs, and then started to spit out the same error. Also, some other similar modules, like |
|
I have experienced the same bug when connecting from a Debian unstable system (Debian mitogen 0.3.1-3) to a Debian 11 system. I fixed it by installing on the target these packages (I did not check which one was actually relevant): python3-chardet python3-decorator python3-pkg-resources python3-requests. |
|
Same here with ansible 4.x and 5.x, mitogen 3.2. I noticed that it happens only on first apt module invocation on fresh clean destination system - python3-apt package installed automatically before actually installing other packages with apt module. It always works on second run and after because python3-apt already installed. So I think task failed because of apt-related files changed during first run (module reuse on something, idk). Confirmed workaround - install python3-apt package on destination system manually or via shell module before first apt module invocation: - name: debian.yml | https://github.com/mitogen-hq/mitogen/issues/849 workaround
shell:
cmd: apt update && apt install -y python3-apt |
|
Hello guys, any news here? Will that be fixed? Thanks. |
|
Having same issue with Ansible + Mitogen :-( |
|
just one consideration: are you checking that the facts are not cached between one experiment and the next? |
|
Notes to self from reproduction attempts. Attempt 1: macOS -> Debian 11.4, python3-apt installed, rev 8cda5f5, Ansible 5, negativePlaybook - name: Module with FQN
hosts: rpi1
become: true
strategy: mitogen_linear
tasks:
- name: install vim
ansible.builtin.apt:
name: vim
state: present
- name: Module without FQN
hosts: rpi1
become: true
strategy: mitogen_linear
tasks:
- name: install vim
apt:
name: vim
state: presentAttempt 2: macOS -> Debian 11.4, python3-apt removed, rev 8cda5f5, Ansible 5, reproduced |
|
In my case, only an upgrade to mitogen 0.3.3 helped when provisioning a Debian 11.4 (5.10.120-1) target system running python 3.9.2. Before that, I had the same problems as described in this thread. current stack of the performing machine
|
|
Same error when running on Ubuntu 18.04.6 LTS. The system has Ubuntu’s Python 3.8 installed and An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: module '__main__' has no attribute '_module_fqn'
fatal: [n0210]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible_mitogen/runner.py\", line 978, in _run\n self._run_code(code, mod)\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible_mitogen/runner.py\", line 942, in _run_code\n exec(code, vars(mod))\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible/modules/apt.py\", line 1387, in <module>\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible/modules/apt.py\", line 1155, in main\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible/module_utils/common/respawn.py\", line 39, in respawn_module\n payload = _create_payload()\n File \"master:/usr/local/lib/python3.8/dist-packages/ansible/module_utils/common/respawn.py\", line 76, in _create_payload\n module_fqn = sys.modules['__main__']._module_fqn\nAttributeError: module '__main__' has no attribute '_module_fqn'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}Ansible Core is v2.12.9, mitogen is v0.3.3. |
|
Any update regarding this? "module_stderr": "Traceback (most recent call last):\n File "master:/runner/project/mitogen-0.3.3/ansible_mitogen/runner.py", line 978, in _run\n self._run_code(code, mod)\n File "master:/runner/project/mitogen-0.3.3/ansible_mitogen/runner.py", line 942, in _run_code\n exec(code, vars(mod))\n File "master:/usr/local/lib/python3.8/site-packages/ansible/modules/dnf.py", line 1427, in \n File "master:/usr/local/lib/python3.8/site-packages/ansible/modules/dnf.py", line 1414, in main\n File "master:/usr/local/lib/python3.8/site-packages/ansible/modules/dnf.py", line 385, in init\n File "master:/usr/local/lib/python3.8/site-packages/ansible/modules/dnf.py", line 578, in _ensure_dnf\n File "master:/usr/local/lib/python3.8/site-packages/ansible/module_utils/common/respawn.py", line 39, in respawn_module\n payload = _create_payload()\n File "master:/usr/local/lib/python3.8/site-packages/ansible/module_utils/common/respawn.py", line 76, in _create_payload\n module_fqn = sys.modules['main']._module_fqn\nAttributeError: module 'main' has no attribute '_module_fqn'\n", Ansible core 2.12.5.post0 It only happens with dnf or yum if are part of the playbook |
In the case of debian, manually installing Perhaps there is a similar analog for CentOS distros as well? |
|
Same here on ubuntu:20.04 with ansible=5.10.0-1ppa~focal (ansible-core 2.12.10) from ppa:ansible/ansible, mitogen 0.3.3 output |
|
I played around a bit to dive deeper into why this We see that ansible wants to respawn the So why would ansible want to respawn this module at all? Looking at the code: def _ensure_dnf(self):
[…]
global dnf
try:
import dnf
import dnf.cli
import dnf.const
import dnf.exceptions
import dnf.subject
import dnf.util
HAS_DNF = True
except ImportError:
HAS_DNF = False
if HAS_DNF:
return
[…]
respawn_module(interpreter)obviously importing I can reproduce the import error easily:
Anyway. This helps to understand what triggers this error, but not how to fix it. Looking at the ansible code we see that the only place In mitogen this error could be fixed by catching exceptions related to As So while this fails at the third tasks:
- dnf: name=foobar state=absent
- dnf: name=glibc state=latest
- dnf: name=vimthis works (for me, ymmv): tasks:
- dnf: name=foobar state=absent
- dnf: name=glibc state=latest
- meta: reset_connection
- dnf: name=vim |
I can confirm this. |
Workaround for mitogen-hq/mitogen#849
|
I confirm this bug still exists. |
|
Same bug with mitogen-0.3.7 |
|
Any workarounds for amazon linux 2 (yum)? |
|
In my situation changing interpreter has fix problem.
...i have this error
|
|
Agree to bazhil. CentOS 7: |
|
Fallback to python2 is not an option guys. |
Yes. But in my situation it helps. Hope in future we find better idea to fix. |
|
Running into this issue on RHEL8 with ansible and mitogen 0.3.9 Adding ansible_python_interpreter doesn't appear to help. Task: Output: |
|
This fully breaks mitogen for me; all yum / package calls from Ansible are broken for specific python versions: Controller node: macOS with ansible [core 2.15.9] and python version = 3.11.10 Failure happens in any ansible call to package/yum, not only the first. I have already moved some tasks to the command module but always the next package or yum ansible call fails. Any ideas? |
moreati
changed the title
|
I've made some progress investigating this in #1162. I think Mitogen will need to implement (& monkeypatch) an version of Ansible's |
moreati
added
the
DesignRequired
|
I'm also experiencing these issues with RHEL8 servers. After upgrading to ansible-core 2.17 and thus requiring Python 3.7+ I installed Python 3.11 on these servers and was unable to run tasks using the dnf module since then. As I have seen @moreati has made some progress in #1162, is there any kind of ETA for this issue? Weeks, months, possibly years? Thanks! |
same thing with updating python 3.6 -> 3.9 on AlmaLinux 8 |
|
@tneidlinger and @badfiles |
|
I'm having the same issue with Ubuntu 20.04 and 22.04 target hosts. It doesn't matter if the requisite python*-apt packages are installed or not; the error is occurring in the respawn part of the local controller (tested on both Mac and Ubuntu controller nodes with matching Pythons). |
|
I have same problem. Python 3.12, ansible 2.16.3, mitogen 0.3.19 |
and others
module_utilsloaded? noExample Playbook:
Error:
Verbose output:
verbose.log.txt
Workarounds like:
vars: { mitogen_task_isolation: fork }to the taskansible.builtin.aptto ALWAYS_FORK_MODULES inansible_mitogen/planner.pydoesn't work.
The text was updated successfully, but these errors were encountered: