Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: switch back to libclang to avoid hardcoding LLVM version #72

Merged
merged 1 commit into from
Oct 14, 2023

Conversation

bgilbert
Copy link
Collaborator

The libclang package has an embedded copy of libclang, which allows us to avoid requiring the system to have a particular clang version. This didn't work before because the lib_search_dirs logic in conf.py was clobbering a working path with a broken one. That logic is no longer needed, so just remove it.

Tested on Fedora 38 and Ubuntu 22.04, but not specifically tested with Read the Docs.

The libclang package has an embedded copy of libclang, which allows us to
avoid requiring the system to have a particular clang version.  This
didn't work before because the lib_search_dirs logic in conf.py was
clobbering a working path with a broken one.  That logic is no longer
needed, so just remove it.
@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

Oh nice. Let's try this!

@jcupitt jcupitt merged commit 662110c into ImagingDataCommons:main Oct 14, 2023
5 checks passed
@bgilbert bgilbert deleted the docs branch October 14, 2023 11:24
@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

Sadly this breaks on readthedocs with what looks like a clang14 / clang16 mismatch error:

clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. 
Please ensure that your python bindings are compatible with your libclang.so version.

The build error report says:

>>> python -m sphinx -T -E -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
Running Sphinx v7.2.6
making output directory... done
WARNING: html_static_path entry '_static' does not exist
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 7 source files that are out of date
updating environment: [new config] 7 added, 0 changed, 0 removed
�[2Kreading sources... [ 14%] api

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4255, in register_function
    func = getattr(lib, item[0])
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/ctypes/__init__.py", line 389, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/ctypes/__init__.py", line 394, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/cmd/build.py", line 298, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 355, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313, in build
    updated_docnames = set(self.read())
                           ^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 420, in read
    self._read_serial(docnames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 441, in _read_serial
    self.read_doc(docname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 498, in read_doc
    publisher.publish()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/io.py", line 105, in read
    self.parse()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/parsers.py", line 81, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
                                  ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
                   ^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
                                  ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/domains/__init__.py", line 289, in run
    return super().run()
           ^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/__init__.py", line 152, in run
    self.__parse(filename)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/__init__.py", line 80, in __parse
    docstrings, errors = parse(filename, domain=self._domain,
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/parser.py", line 755, in parse
    index = Index.create()
            ^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 2875, in create
    return Index(conf.lib.clang_createIndex(excludeDecls, 0))
                 ^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 212, in __get__
    value = self.wrapped(instance)
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4336, in lib
    register_functions(lib, not Config.compatibility_check)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4283, in register_functions
    register(f)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4280, in register
    return register_function(lib, item, ignore_errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4261, in register_function
    raise LibclangError(msg)
clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. Please ensure that your python bindings are compatible with your libclang.so version.

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4261, in register_function
    raise LibclangError(msg)
clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. Please ensure that your python bindings are compatible with your libclang.so version.
The full traceback has been saved in /tmp/sphinx-err-19649r46.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

Maybe requirement.txt needs libclang <= 14?

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

Recent readthedocs builds are here: https://readthedocs.org/projects/libdicom/builds/

@bgilbert
Copy link
Collaborator Author

Oof. I wonder why it works on vanilla 22.04 and fails on ReadTheDocs. Maybe we can use a separate requirements.txt for ReadTheDocs? I don't think it's a good idea to hardcode an LLVM 14 requirement in the Meson build, because the docs need to be buildable on other distros as well. E.g., Hawkmoth is in the process of being packaged for Fedora, and once it lands, the Fedora libdicom package will rebuild the docs from source.

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

Yes, it's a bit annoying, I spent a morning getting readthedocs working :( I wish it was simpler.

I'll revert this commit for now to get the docs building again.

@jcupitt
Copy link
Collaborator

jcupitt commented Oct 14, 2023

They have their build images here, fwiw: https://github.com/readthedocs/readthedocs-docker-images

@bgilbert
Copy link
Collaborator Author

Second attempt in #74.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants