Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: enable WAMR and WAVM runtimes. #162

Merged
merged 27 commits into from
May 16, 2021
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9337c10
build: enable WAMR and LLVM builds.
mathetake May 12, 2021
fe09128
Add WAVM.
mathetake May 12, 2021
26a0890
Use clang.
mathetake May 12, 2021
8357bea
Fix test path
mathetake May 12, 2021
9d15fb0
Cache on runtime dependency versions.
mathetake May 12, 2021
8296f8e
Fix WAVM build and disable runtime_test temporarily.
mathetake May 12, 2021
cf4978d
Fix bazelrc.
mathetake May 12, 2021
0e077c4
Fix runtime name.
mathetake May 12, 2021
5770ad4
review: add Ninja options.
mathetake May 12, 2021
6abbc2d
Merge remote-tracking branch 'origin/master' into llvm-wamr
mathetake May 12, 2021
c67261f
review: remove deprecated options, LLVM patch.
mathetake May 13, 2021
117534a
Use cache@v2 for better compression
mathetake May 13, 2021
db4b0ce
Fix runtime tests.
mathetake May 13, 2021
180cfd0
Fix WAVM build with unresolved TODO.
mathetake May 13, 2021
72a8153
review: use simple cache key.
mathetake May 13, 2021
f3d9496
Update WAMR and use Ninja.
mathetake May 13, 2021
90dc748
review: Delete trap2 test.
mathetake May 13, 2021
0bb696f
Add missing header in NullVm.
mathetake May 13, 2021
cb274d3
review: restore seprate trap2 test with TODO for WAVM and fix nits.
mathetake May 13, 2021
86002c8
Merge remote-tracking branch 'origin/master' into llvm-wamr
mathetake May 13, 2021
6fed6fe
review: fix style.
mathetake May 14, 2021
502c42e
try removing -v.
mathetake May 14, 2021
7492878
review: disable LLVM benchmark build.
mathetake May 15, 2021
cb4a607
Merge branch 'master' into llvm-wamr
PiotrSikora May 15, 2021
7822e89
Fix merge (missing new line at the end).
PiotrSikora May 15, 2021
d6ba6ab
review: use --per_file_copt.
PiotrSikora May 15, 2021
6cf8d7b
review: wipe cache.
PiotrSikora May 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
build --action_env=CC=clang
build --action_env=CXX=clang++
build --enable_platform_specific_config
mathetake marked this conversation as resolved.
Show resolved Hide resolved

build:linux --cxxopt=-std=c++17
Expand Down
25 changes: 21 additions & 4 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,23 +55,40 @@ jobs:
addlicense -check .

build:
name: build (${{ matrix.runtime }})

runs-on: ubuntu-latest

strategy:
matrix:
mathetake marked this conversation as resolved.
Show resolved Hide resolved
# TODO(mathetake): Add other runtimes.
runtime: [ "wasmtime" ]
# runtime: Wasm runtime.
# cache_key_regex: used for calculating Bazel cache key.
include:
# TODO(mathetake): Add V8.
- runtime: wamr
cache_key_regex: bytecodealliance/wasm-micro-runtime|llvm-project
- runtime: wasmtime
cache_key_regex: bytecodealliance/wasmtime
- runtime: wavm
cache_key_regex: WAVM/WAVM|llvm-project

steps:
- uses: actions/checkout@v2

# Calculate the cache key by greping target repositories in repositories.bzl with "matrix.cache_key_regex".
# TODO(mathetake): Use native Bazel solution if possible.
- name: Setup Bazel cache key
run: |
GHA_BAZEL_CACHE_KEY=$(cat bazel/repositories.bzl | grep -E "${{ matrix.cache_key_regex }}" | sha1sum)
echo "GHA_BAZEL_CACHE_KEY=${GHA_BAZEL_CACHE_KEY}" >> $GITHUB_ENV

- name: Mount bazel cache
mathetake marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/cache@v1
with:
path: "/home/runner/.cache/bazel"
mathetake marked this conversation as resolved.
Show resolved Hide resolved
key: bazel-${{ matrix.runtime }}
key: bazel-${{ env.GHA_BAZEL_CACHE_KEY }}

- name: Test
run: |
bazel test --define runtime=${{ matrix.runtime }} //...
bazel test --define runtime=${{ matrix.runtime }} //test/...

19 changes: 11 additions & 8 deletions BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,12 @@ cc_library(
cc_library(
name = "wamr_lib",
srcs = glob([
# TODO(@mathetake): Add WAMR lib.
# "src/wamr/*.h",
# "src/wamr/*.cc",
"src/wamr/*.h",
"src/wamr/*.cc",
]),
deps = [
":common_lib",
# TODO(@mathetake): Add WAMR lib.
"@wamr//:wamr_lib",
],
)

Expand All @@ -79,13 +78,17 @@ cc_library(
cc_library(
name = "wavm_lib",
srcs = glob([
# TODO(@mathetake): Add WAVM lib.
# "src/wavm/*.h",
# "src/wavm/*.cc",
"src/wavm/*.h",
"src/wavm/*.cc",
]),
copts = [
'-DWAVM_API=""',
"-Wno-non-virtual-dtor",
"-Wno-old-style-cast",
],
deps = [
":common_lib",
# TODO(@mathetake): Add WAVM lib.
"@wavm//:wavm_lib",
],
)

Expand Down
112 changes: 112 additions & 0 deletions bazel/external/llvm.BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")

licenses(["notice"]) # Apache 2

package(default_visibility = ["//visibility:public"])

filegroup(
name = "srcs",
srcs = glob(["**"]),
)

cmake(
name = "llvm_lib",
mathetake marked this conversation as resolved.
Show resolved Hide resolved
cache_entries = {
# Disable both: BUILD and INCLUDE, since some of the INCLUDE
# targets build code instead of only generating build files.
"LLVM_BUILD_DOCS": "off",
"LLVM_INCLUDE_DOCS": "off",
"LLVM_BUILD_EXAMPLES": "off",
"LLVM_INCLUDE_EXAMPLES": "off",
PiotrSikora marked this conversation as resolved.
Show resolved Hide resolved
"LLVM_BUILD_RUNTIME": "off",
"LLVM_BUILD_RUNTIMES": "off",
"LLVM_INCLUDE_RUNTIMES": "off",
"LLVM_BUILD_TESTS": "off",
"LLVM_INCLUDE_TESTS": "off",
"LLVM_BUILD_TOOLS": "off",
"LLVM_INCLUDE_TOOLS": "off",
"LLVM_BUILD_UTILS": "off",
"LLVM_INCLUDE_UTILS": "off",
"LLVM_ENABLE_LIBEDIT": "off",
"LLVM_ENABLE_LIBXML2": "off",
"LLVM_ENABLE_TERMINFO": "off",
"LLVM_ENABLE_ZLIB": "off",
"LLVM_TARGETS_TO_BUILD": "X86",
# Workaround for the issue with statically linked libstdc++
# using -l:libstdc++.a.
"CMAKE_CXX_FLAGS": "-lstdc++",
},
env_vars = {
# Workaround for the -DDEBUG flag added in fastbuild on macOS,
# which conflicts with DEBUG macro used in LLVM.
"CFLAGS": "-UDEBUG",
"CXXFLAGS": "-UDEBUG",
"ASMFLAGS": "-UDEBUG",
},
lib_source = ":srcs",
out_static_libs = [
"libLLVMInterpreter.a",
"libLLVMWindowsManifest.a",
"libLLVMLibDriver.a",
"libLLVMObjectYAML.a",
"libLLVMCoverage.a",
"libLLVMLineEditor.a",
"libLLVMDlltoolDriver.a",
"libLLVMOption.a",
"libLLVMTableGen.a",
"libLLVMFuzzMutate.a",
"libLLVMSymbolize.a",
"libLLVMCoroutines.a",
"libLLVMDebugInfoPDB.a",
"libLLVMLTO.a",
"libLLVMObjCARCOpts.a",
"libLLVMMIRParser.a",
"libLLVMOrcJIT.a",
"libLLVMOrcError.a",
"libLLVMJITLink.a",
"libLLVMPasses.a",
"libLLVMipo.a",
"libLLVMInstrumentation.a",
"libLLVMVectorize.a",
"libLLVMLinker.a",
"libLLVMIRReader.a",
"libLLVMAsmParser.a",
"libLLVMX86Disassembler.a",
"libLLVMX86AsmParser.a",
"libLLVMX86CodeGen.a",
"libLLVMCFGuard.a",
"libLLVMGlobalISel.a",
"libLLVMSelectionDAG.a",
"libLLVMAsmPrinter.a",
"libLLVMDebugInfoDWARF.a",
"libLLVMCodeGen.a",
"libLLVMScalarOpts.a",
"libLLVMInstCombine.a",
"libLLVMAggressiveInstCombine.a",
"libLLVMTransformUtils.a",
"libLLVMBitWriter.a",
"libLLVMX86Desc.a",
"libLLVMMCDisassembler.a",
"libLLVMX86Utils.a",
"libLLVMX86Info.a",
"libLLVMMCJIT.a",
"libLLVMExecutionEngine.a",
"libLLVMTarget.a",
"libLLVMAnalysis.a",
"libLLVMProfileData.a",
"libLLVMRuntimeDyld.a",
"libLLVMObject.a",
"libLLVMTextAPI.a",
"libLLVMMCParser.a",
"libLLVMBitReader.a",
"libLLVMMC.a",
"libLLVMDebugInfoCodeView.a",
"libLLVMDebugInfoMSF.a",
"libLLVMCore.a",
"libLLVMRemarks.a",
"libLLVMBitstreamReader.a",
"libLLVMBinaryFormat.a",
"libLLVMSupport.a",
"libLLVMDemangle.a",
],
)
25 changes: 25 additions & 0 deletions bazel/external/llvm.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Workaround for Envoy's CMAKE_BUILD_TYPE=Bazel.
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -247,7 +247,7 @@
string(TOUPPER "${CMAKE_BUILD_TYPE}" uppercase_CMAKE_BUILD_TYPE)

if (CMAKE_BUILD_TYPE AND
- NOT uppercase_CMAKE_BUILD_TYPE MATCHES "^(DEBUG|RELEASE|RELWITHDEBINFO|MINSIZEREL)$")
+ NOT uppercase_CMAKE_BUILD_TYPE MATCHES "^(DEBUG|RELEASE|RELWITHDEBINFO|MINSIZEREL|BAZEL)$")
mathetake marked this conversation as resolved.
Show resolved Hide resolved
message(FATAL_ERROR "Invalid value for CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
endif()

# Workaround for a missing -fuse-ld flag in CXXFLAGS, which results in
# different linkers being used during configure and compilation phases.
--- a/cmake/modules/HandleLLVMOptions.cmake
+++ b/cmake/modules/HandleLLVMOptions.cmake
@@ -718,8 +718,6 @@ endif()
if (UNIX AND CMAKE_GENERATOR STREQUAL "Ninja")
include(CheckLinkerFlag)
check_linker_flag("-Wl,--color-diagnostics" LINKER_SUPPORTS_COLOR_DIAGNOSTICS)
- append_if(LINKER_SUPPORTS_COLOR_DIAGNOSTICS "-Wl,--color-diagnostics"
- CMAKE_EXE_LINKER_FLAGS CMAKE_MODULE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS)
endif()

# Add flags for add_dead_strip().
2 changes: 2 additions & 0 deletions bazel/external/proxy-wasm-cpp-sdk.BUILD
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
load("@rules_cc//cc:defs.bzl", "cc_library")

licenses(["notice"]) # Apache 2

package(default_visibility = ["//visibility:public"])
Expand Down
15 changes: 10 additions & 5 deletions bazel/external/wamr.BUILD
Original file line number Diff line number Diff line change
@@ -1,25 +1,30 @@
licenses(["notice"]) # Apache 2

load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")

licenses(["notice"]) # Apache 2

package(default_visibility = ["//visibility:public"])

filegroup(
name = "srcs",
srcs = glob(["**"]),
visibility = ["//visibility:public"],
)

cmake(
name = "libiwasm",
name = "wamr_lib",
cache_entries = {
"LLVM_DIR": "$EXT_BUILD_DEPS/copy_llvm/llvm/lib/cmake/llvm",
"WAMR_BUILD_INTERP": "1",
"WAMR_BUILD_JIT": "0",
"WAMR_BUILD_AOT": "0",
"WAMR_BUILD_SIMD": "0",
"WAMR_BUILD_MULTI_MODULE": "1",
"WAMR_BUILD_LIBC_WASI": "0",
"WAMR_BUILD_TAIL_CALL": "1",
},
defines = ["WASM_WAMR"],
lib_source = ":srcs",
out_shared_libs = ["libiwasm.so"],
out_static_libs = ["libvmlib.a"],
deps = [
"@llvm//:llvm_lib",
],
)
2 changes: 2 additions & 0 deletions bazel/external/wasm-c-api.BUILD
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
load("@rules_cc//cc:defs.bzl", "cc_library")

licenses(["notice"]) # Apache 2

package(default_visibility = ["//visibility:public"])
Expand Down
1 change: 1 addition & 0 deletions bazel/external/wasmtime.BUILD
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
load("@rules_cc//cc:defs.bzl", "cc_library")
load("@rules_rust//rust:rust.bzl", "rust_library")

licenses(["notice"]) # Apache 2
Expand Down
38 changes: 38 additions & 0 deletions bazel/external/wavm.BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")

licenses(["notice"]) # Apache 2

package(default_visibility = ["//visibility:public"])

filegroup(
name = "srcs",
srcs = glob(["**"]),
)

cmake(
name = "wavm_lib",
binaries = ["wavm"],
mathetake marked this conversation as resolved.
Show resolved Hide resolved
cache_entries = {
"LLVM_DIR": "$EXT_BUILD_DEPS/copy_llvm/llvm/lib/cmake/llvm",
"WAVM_ENABLE_STATIC_LINKING": "on",
"WAVM_ENABLE_RELEASE_ASSERTS": "on",
"WAVM_ENABLE_UNWIND": "on",
# Workaround for the issue with statically linked libstdc++
# using -l:libstdc++.a.
"CMAKE_CXX_FLAGS": "-lstdc++ -Wno-unused-command-line-argument",
},
defines = ["WASM_WAVM"],
env_vars = {
# Workaround for the -DDEBUG flag added in fastbuild on macOS,
# which conflicts with DEBUG macro used in LLVM.
"CFLAGS": "-UDEBUG",
"CXXFLAGS": "-UDEBUG",
"ASMFLAGS": "-UDEBUG",
},
lib_source = ":srcs",
static_libraries = [
mathetake marked this conversation as resolved.
Show resolved Hide resolved
"libWAVM.a",
"libWAVMUnwind.a",
],
deps = ["@llvm//:llvm_lib"],
)
18 changes: 18 additions & 0 deletions bazel/repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,21 @@ def proxy_wasm_cpp_host_repositories():
strip_prefix = "rules_foreign_cc-0.2.0",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/0.2.0.tar.gz",
)

http_archive(
name = "llvm",
build_file = "@proxy_wasm_cpp_host//bazel/external:llvm.BUILD",
sha256 = "df83a44b3a9a71029049ec101fb0077ecbbdf5fe41e395215025779099a98fdf",
strip_prefix = "llvm-10.0.0.src",
url = "https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/llvm-10.0.0.src.tar.xz",
patch_args = ["-p1"],
patches = ["@proxy_wasm_cpp_host//bazel/external:llvm.patch"],
)

http_archive(
name = "wavm",
build_file = "@proxy_wasm_cpp_host//bazel/external:wavm.BUILD",
sha256 = "ce899269516313b400005a8cc9bc3bcd8329663f43f7b4baae211ea0cd456a39",
strip_prefix = "WAVM-79c3aa29366615d9b1593cd527e5b4b94cc6072a",
url = "https://github.com/WAVM/WAVM/archive/79c3aa29366615d9b1593cd527e5b4b94cc6072a.tar.gz",
)
30 changes: 15 additions & 15 deletions test/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ cc_test(
],
)

cc_test(
name = "runtime_test",
srcs = ["runtime_test.cc"],
data = [
"//test/test_data:abi_export.wasm",
"//test/test_data:callback.wasm",
"//test/test_data:trap.wasm",
],
deps = [
":utility_lib",
"//:lib",
"@com_google_googletest//:gtest",
"@com_google_googletest//:gtest_main",
],
)
# cc_test(
# name = "runtime_test",
# srcs = ["runtime_test.cc"],
# data = [
# "//test/test_data:abi_export.wasm",
# "//test/test_data:callback.wasm",
# "//test/test_data:trap.wasm",
# ],
# deps = [
# ":utility_lib",
# "//:lib",
# "@com_google_googletest//:gtest",
# "@com_google_googletest//:gtest_main",
# ],
# )

cc_test(
name = "exports_test",
Expand Down