build: Add support for no_xdp and no_xdp_tbs mode

Previously, the xdp existence and xdp_tbs existence is required for tsn ref sw build.
Now, we are adding additional checks to support non-xdp or non-xdp_tbs system.

*XDP existence is based on if_xdp.h existence
*XDP_TBS based on txtime member if_xdp.h:xdp_desc structure

1. If if_xdp.h is not there, the XDP(and XDP_TBS as well) is considered as unsupported.
   In this case, opcua-server will not be compiled.
2. Table of current supported application according to if_xdp.h availability on the system.
   if_xdp.h indicates that the XDP socket is available to the applications side.
   Without if_xdp.h only AF_PACKET-socket-based (tsq & txrx-tsn) test will be offered.

                                                 |    tsq | txrx-tsn  | opcua-server  |
    -----------------------------------------------------------------------------------
    WITHOUT XDP                                  |     Y  |     Y     |     N         |
    - if_xdp.h not detected                      |        |           |               |
    -----------------------------------------------------------------------------------

2. Table of current supported application according to xdp_tbs availability on the system.

                                                 |    tsq | txrx-tsn  | opcua-server  |
    -----------------------------------------------------------------------------------
    WITH XDP_TBS                                 |     Y  |     Y     |     Y         |
    - if_xdp.h detected (WITH_XDP)               |        |           |               |
    - XDP_TBS mode not disabled                  |        |           |               |
    - txtime var is detected in if_xdp.h         |        |           |               |
    -----------------------------------------------------------------------------------
    WITHOUT XDP_TBS                              |      Y |       Y   |     N(WIP)    |
    - if_xdp.h detected (WITH_XDP)               |        |           |               |
    - XDP_TBS mode disabled (-t) or              |        |           |               |
    - txtime var is NOT detected in if_xdp.h     |        |           |               |
    -----------------------------------------------------------------------------------

Default build (NO -t) is enabling XDP feature and Intel-specific XDP+TBS feature (if they are supported !)

Note: Effort is ongoing to decouple libopen62541-iotg fork and opcua-server from XDP_TBS Intel implementation.

Signed-off-by: Noor Azura Ahmad Tarmizi <noor.azura.ahmad.tarmizi@intel.com>

DEPENDENCIES.md

@@ -5,19 +5,19 @@ software dependencies would have already been installed.
 
 For compilation:
 * [Custom linux kernel headers](https://github.com/intel/linux-intel-lts/tree/5.4/preempt-rt)
+  - Include support for Intel XDP+TBS implementation
 * [Custom linux-libc-headers](https://github.com/intel/iotg-yocto-ese-bsp/tree/master/recipes-kernel/linux-libc-headers/linux-libc-headers)
+  - Include support for Intel XDP+TBS implementation
 * [Custom libopen62541-iotg](https://github.com/intel/iotg-yocto-ese-main/tree/master/recipes-connectivity/open62541)
+  - Include support for kernel implementation of Intel XDP+TBS (if_xdp.h change required)
 * [Custom libbpf](https://github.com/intel/iotg-yocto-ese-main/tree/master/recipes-connectivity/libbpf)
+  - Include support for kernel implementation of Intel XDP+TBS (if_xdp.h change required)
 * libelf
 * libjson-c
 
-In order to compile under Ubuntu, these packages need to be installed first:
-* gcc
-* autoconf
-* libjson-c-dev
-* gawk
-* build-essential
-Note : Customized libbpf and libopen62541-iotg is a hard dependencies as per now (sorry). WIP to decouple tsn ref sw from the customized dependencies
+**NOTE**
+Customized libbpf and libopen62541-iotg is a hard dependencies as per now (sorry). WIP to decouple tsn ref sw from the customized dependencies.
+See below on how to install the customized libbpf and libopen62541-iotg.
 
 For run-time:
 * [Custom linux kernel](https://github.com/intel/linux-intel-lts/tree/5.4/preempt-rt)
@@ -28,6 +28,33 @@ For run-time:
 * gnuplot 5.2
 * IceWM - Any GUI/window manager can be used, required to display graphs.
 
+## Ubuntu based dependencies
+In order to compile under Ubuntu, these packages need to be installed first:
+* gcc
+* autoconf
+* libjson-c-dev
+* gawk
+* build-essential
+
+## Dependencies installer script
+In order to ease the installation of the customized helper libraries needed, we have included a few installation scripts in a specific branch.
+These scripts will check out a specific version of the libraries from upstream git and apply our patches on top of it.
+
+Branch name : **experimental/dependencies_installer**
+The dependencies installer branch : https://github.com/intel/iotg_tsn_ref_sw/tree/experimental/dependencies_installer/
+
+Refer to the README.md to understand the differences between the scripts provided:
+
+https://github.com/intel/iotg_tsn_ref_sw/blob/experimental/dependencies_installer/dependencies/README.md
+
+Suggestion is to use the overwriting installer script (after having kept the original libbpf and libopen62541 in a safe place).
+This will ensure the tsn ref sw will be able to find/use correct libraries.
+
+NOTE: If you are installing the libbpf and libopen62541-iotg manually, there is a chance the related open62541-iotg.pc file is not there. Hence, you might have to comment out this line in **configure.ac**.
+
+**AM_CONDITIONAL([WITH_OPCUA], [test "x$no_open62541_iotg_fork" = "x0"] && test "x${HAVE_XDP}" = "xyes")**
+
+
 # Hardware dependencies
 
 * Refer to compatible platform list.

Makefile.am

@@ -1,6 +1,21 @@
-bin_PROGRAMS = tsq txrx-tsn opcua-server
+bin_PROGRAMS = tsq txrx-tsn
+
+if WITH_OPCUA
+bin_PROGRAMS += opcua-server
+endif
+
 tsq_SOURCES = src/tsq.c
-txrx_tsn_SOURCES = src/txrx.c src/txrx-afpkt.c src/txrx-afxdp.c
+
+txrx_tsn_SOURCES = src/txrx.c src/txrx-afpkt.c
+
+if WITHXDP
+txrx_tsn_SOURCES += src/txrx-afxdp.c
+endif
+
+if ! WITHXDPTBS
+EXTRA_CFLAGS_NOXDPTBS = -Wno-unused-but-set-parameter -Wunused-but-set-variable
+endif
+
 opcua_server_SOURCES=src/opcua-tsn/multicallback_server.c   \
 			src/opcua-tsn/json_helper.c	\
 			src/opcua-tsn/opcua_common.c	\
@@ -14,5 +29,5 @@ opcua_server_LDADD = $(open62451_LIBS) $(libjson_LIBS) $(libbpf_LIBS) $(libelf_L
 AM_CPPFLAGS = -O2 -g -fstack-protector-strong -fPIE -fPIC -D_FORTIFY_SOURCE=2 \
 		-Wformat -Wformat-security -Wformat-overflow -Wno-parentheses \
 		-Wno-missing-field-initializers -Wextra -Wall -fno-common \
-		$(open62451_CFLAGS) $(libjson_CFLAGS) $(libbpf_CFLAGS) $(libelf_CFLAGS)
+		$(open62451_CFLAGS) $(libjson_CFLAGS) $(libbpf_CFLAGS) $(libelf_CFLAGS) $(ENABLEXDP_CPPFLAGS) $(EXTRA_CFLAGS_NOXDPTBS)
 AM_LDFLAGS = -Wl,-z,noexecstack,-z,relro,-z,now -pie

README.md

@@ -71,6 +71,8 @@ For further configuration details, check out [README_config.md](README_config.md
   * [Platform specificity](README_project.md#platform-specificity)
   * [Optimization](README_project.md#optimization)
   * [Role of run.sh](README_project.md#role-of-runsh)
+* [Installation](README_install.md)
+* [Tsn Ref Sw Dependencies](DEPENDENCIES.md)
 * [File Structure & Conventions](README_project.md#file-structure-conventions)
 * [Configuration](README_config.md)
 * [Shell-based runner](README_shell.md)

README_install.md

@@ -0,0 +1,82 @@
+# Installation HOWTO
+
+## Tsn ref sw build and installation
+
+To build tsn ref sw, we are currently providing a single script that will build all
+binaries (tsq, txrx-tsn and opcua-server).
+
+    ```sh
+    cd <tsn_ref_sw_directory>
+    ./build.sh <-h|-v|-V|-x|-t>
+    ```
+
+    **Build option**
+
+    ```/usr/share/iotg_tsn_ref_sw# ./build.sh -h
+
+        BUILD.SH: Checking Default Shell
+        Default Shell: sh (Valid)
+        Usage: ./build.sh [-hxtVv]
+
+        -h,     -help,             --help             Display help
+
+        -t,     -disablexdptbs,    --disablexdptbs    Disable Intel specific XDP+TBS support. (By default XDP+TBS support is enabled)
+
+        -V,     -verbose,          --verbose          Run script in verbose mode.
+
+        -v,     -version,          --version          Show version.
+    ```
+
+Default build (NO build options passed) is enabling XDP feature and Intel-specific XDP+TBS feature (if supported (if_xdp.h check)).
+
+### To explicitly disable Intel specific XDP+TBS support in tsn ref sw
+
+    ```sh
+    cd <tsn_ref_sw_directory>
+    ./build.sh -t
+    ```
+
+### Check version
+
+    ```sh
+    cd <tsn_ref_sw_directory>
+    ./build.sh -v
+    ```
+
+NOTE
+
+1. If if_xdp.h is not there, the XDP (and XDP_TBS as well) is considered as unsupported.
+   In this case, opcua-server will not be compiled.
+2. Table of current supported application according to if_xdp.h availability on the system.
+   if_xdp.h indicates that the XDP socket is available to the applications side.
+   Without if_xdp.h, only AF_PACKET-socket-based (tsq & txrx-tsn) test will be offered.
+
+                                                 |    tsq | txrx-tsn  | opcua-server  |
+    -----------------------------------------------------------------------------------
+    WITHOUT XDP                                  |     Y  |     Y     |     N         |
+    - if_xdp.h not detected                      |        |           |               |
+    -----------------------------------------------------------------------------------
+
+2. Table of current supported application according to xdp_tbs availability on the system.
+
+
+                                                 |    tsq | txrx-tsn  | opcua-server  |
+    -----------------------------------------------------------------------------------
+    WITH XDP_TBS                                 |     Y  |     Y     |     Y         |
+    - if_xdp.h detected (WITH_XDP)               |        |           |               |
+    - XDP_TBS mode not disabled                  |        |           |               |
+    - txtime var is detected in if_xdp.h         |        |           |               |
+    -----------------------------------------------------------------------------------
+    WITHOUT XDP_TBS                              |      Y |       Y   |     N(WIP)    |
+    - if_xdp.h detected (WITH_XDP)               |        |           |               |
+    - XDP_TBS mode disabled (-t) or              |        |           |               |
+    - txtime var is NOT detected in if_xdp.h     |        |           |               |
+    -----------------------------------------------------------------------------------
+
+Default build (NO -t) is enabling XDP feature and Intel-specific XDP+TBS feature (if XDP+TBS is supported in the system!)
+
+Note: Effort is ongoing to decouple libopen62541-iotg fork and opcua-server from XDP_TBS Intel implementation.
+
+## Tsn sw ref dependencies installation
+
+Refer to [Tsn Ref Sw Dependencies](DEPENDENCIES.md)

build.sh

@@ -30,6 +30,25 @@
 #  POSSIBILITY OF SUCH DAMAGE.
 # *****************************************************************************/
 
+showTsnRefHelp() {
+cat << EOF
+Usage: ./build.sh [-hxtVv]
+
+-h,     -help,             --help             Display help
+
+-t,     -disablexdptbs,    --disablexdptbs    Disable Intel specific XDP+TBS support. (By default XDP+TBS support is enabled)
+
+-V,     -verbose,   --verbose   Run script in verbose mode.
+
+-v,     -version,   --version   Show version.
+
+EOF
+}
+
+showTsnRefVersion() {
+    sed -n '/^TSNREFSW_PACKAGE_VERSION/p' ./run.sh
+}
+
 # Check for default shell
 echo -e "\nBUILD.SH: Checking Default Shell"
 default_shell=$(echo $(realpath /usr/bin/sh) | cut -c 10-)
@@ -41,6 +60,45 @@ else
     exit 1
 fi
 
+# By default we enable xdp and xdp+tsb(when supported) intel specific implementation.
+export version=0
+export verbose=0
+export ENABLE_XDP="--enable-xdp"
+export ENABLE_XDPTBS="--enable-xdptbs"
+
+options=$(getopt -l "help,disablexdptbs,verbose,version" -o "htVv" -a -- "$@")
+eval set -- "$options"
+
+while true
+do
+case $1 in
+-h|--help)
+    showTsnRefHelp
+    exit 0
+    ;;
+-v|--version)
+    showTsnRefVersion
+    exit 0
+    ;;
+-t|--disablexdptbs)
+    export ENABLE_XDPTBS=""
+    ;;
+-V|--verbose)
+    export verbose=1
+    set -xv # Set xtrace and verbose mode
+    ;;
+--)
+    shift
+    break;;
+esac
+shift
+done
+
+echo "./build.sh settings:"
+echo "1. verbose = $verbose"
+echo "2. enablexdp = $ENABLE_XDP"
+echo "3. enablexdptbs = $ENABLE_XDPTBS"
+
 touch Makefile.am configure.ac
 
 # Ideally, just use git reset HEAD --hard
@@ -67,7 +125,8 @@ rm -rf  Makefile                \
 
 echo -e "\nBUILD.SH: Configure"
 autoreconf --install
-./configure --prefix /usr --with-open62541-headers=/usr/include/open62541
+./configure --prefix /usr $ENABLE_XDP $ENABLE_XDPTBS
+
 if [ $? -ne 0 ]; then echo "Configure failed"; exit 1; fi
 
 echo -e "\nBUILD.SH: Compile"

configure.ac

@@ -2,14 +2,107 @@ AC_INIT([iotg-tsn-ref-sw], [1.0])
 AM_INIT_AUTOMAKE([-Wall -Werror foreign subdir-objects])
 AC_PROG_CC
 
-PKG_CHECK_MODULES([libelf], [libelf])
-PKG_CHECK_MODULES([libbpf], [libbpf])
-PKG_CHECK_MODULES([libjson], [json-c])
-
-# Point to the installed iotg fork of open62541 library.
-# Refer to link below for the patches & commit used to creat this fork:
-# https://github.com/intel/iotg-yocto-ese-main/tree/master/recipes-connectivity/open62541
-PKG_CHECK_MODULES([open62451], [open62541-iotg])
+# Check for XDP header file
+AC_CHECK_HEADERS([linux/if_xdp.h], [HAVE_XDP=yes], [HAVE_XDP=no])
+
+HAVE_XDPTBS=no
+if test "x${HAVE_XDP}" = "xno"; then
+  AC_MSG_WARN([if_xdp.h is not found in the system. AF_XDP is considered as un-supported.])
+  AC_MSG_NOTICE([Opcua-server will not be compiled because AF_XDP is mandatory for it. Only txrx-tsn and tsq will be compiled.])
+else
+  # Check if XDP+TBS Intel specific kernel implementation is available (the extra txtime var in xdp_desc)
+  AC_CHECK_MEMBER([struct xdp_desc.txtime],[HAVE_XDPTBS=yes],
+                [AC_MSG_WARN([txtime is not found in if_xdp.h:xdp_desc. Intel XDP+TBS is not supported!])],
+                [[#include "linux/if_xdp.h"]])
+fi
+
+# We can only enable xdp if we support XDP in the system (if_xdp.h exists)
+AC_ARG_ENABLE([xdp],
+              AS_HELP_STRING([--enable-xdp], [enable xdp[default=yes]]))
+AC_MSG_CHECKING([whether we choose to enable socket xdp])
+AS_IF([test "x${enable_xdp}" = "xyes"], AC_MSG_RESULT([yes]), AC_MSG_RESULT([no]))
+
+AS_IF([test "x${enable_xdp}" = "xyes"],
+  [AS_IF([test "x${HAVE_XDP}" = "xyes"], [use_xdp=yes], [use_xdp=no])],
+  [use_xdp=no])
+
+AC_MSG_CHECKING([whether the system supports xdp socket and we are enabling xdp socket usage])
+AS_IF([test "x${use_xdp}" = "xyes"], AC_MSG_RESULT([yes]), AC_MSG_RESULT([no]))
+AM_CONDITIONAL([WITHXDP], [test "x${use_xdp}" = "xyes"])
+
+# We can only enable xdp_tbs if we support XDP_tbs in the system (if_xdp.h txtime var exists)
+AC_ARG_ENABLE(xdptbs,[AS_HELP_STRING([--enable-xdptbs], [enable xdptbs])])
+AC_MSG_CHECKING([whether we choose to enable xdp+tbs feature])
+AS_IF([test "x${enable_xdptbs}" = "xyes" ], AC_MSG_RESULT([yes]), AC_MSG_RESULT([no]))
+
+AS_IF([test "x${enable_xdptbs}" = "xyes"],
+  [AS_IF([test "x${use_xdp}" = "xyes"],
+      [AS_IF([test "x${HAVE_XDPTBS}" = "xyes"],[use_xdptbs=yes],[use_xdptbs=no])],
+      [use_xdptbs=no])],
+  [use_xdptbs=no])
+
+AC_MSG_CHECKING([whether the system supports xdp+tbs and we are enabling xdp+tbs feature])
+AS_IF([test "x${use_xdptbs}" = "xyes" ], AC_MSG_RESULT([yes]), AC_MSG_RESULT([no]))
+AM_CONDITIONAL([WITHXDPTBS], [test "x${use_xdptbs}" = "xyes"])
+
+if test "x${use_xdp}" = "xyes"; then
+  if test "x${use_xdptbs}" = "xyes"; then
+    ENABLEXDP_CPPFLAGS="-DWITH_XDP -DWITH_XDPTBS"
+  else
+    ENABLEXDP_CPPFLAGS="-DWITH_XDP"
+  fi
+else
+  ENABLEXDP_CPPFLAGS=""
+fi
+
+AC_SUBST(ENABLEXDP_CPPFLAGS)
+
+#TODO check using AC_SEARCH_LIBS instead of PKG_CHECK_MODULES in the next iteration.
+PKG_CHECK_MODULES([libelf], [libelf],,[AC_MSG_WARN([libelf.pc is not found in the system. Pls ensure the lib is installed anyway!!])])
+PKG_CHECK_MODULES([libbpf], [libbpf],,[AC_MSG_WARN([libbpf.pc is not found in the system. Pls ensure the lib is installed anyway!!])])
+PKG_CHECK_MODULES([libjson], [json-c],,[AC_MSG_WARN([json-c.pc is not found in the system. Pls ensure the lib is installed anyway!!])])
+
+no_open62541_iotg_fork=0
+
+PKG_CHECK_MODULES([open62451],
+    [open62541-iotg],
+    [AC_DEFINE([HAVE_OPEN62541_IOTG], [1], [Using open62541 using iotg fork])],
+    [no_open62541_iotg_fork=1])
+
+if test "x${no_open62541_iotg_fork}" = "x1"; then
+  AC_MSG_WARN([open62541-iotg.pc is not found in the system, libopen62541-iotg is considered as uninstalled. Opcua-based test(opcua-server) will NOT be compiled])
+  AC_MSG_NOTICE([For libopen62541-iotg manual installation, refer to DEPENDENCIES.md])
+fi
+
+AM_CONDITIONAL([NO_OPEN62541_IOTG_LIB], [test "x$no_open62541_iotg_fork" = "x1"])
+# Support for af_xdp (if_xdp.h must be in) is mandatory for opcua-server
+AM_CONDITIONAL([WITH_OPCUA], [test "x${no_open62541_iotg_fork}" = "x0" && test "x${HAVE_XDP}" = "xyes"])
+
+if test "x${no_open62541_iotg_fork}" = "x0"; then
+  if test "x${HAVE_XDP}" = "xno"; then
+    AC_MSG_WARN([Without XDP support, opcua-server app will NOT be compiled (for now)])
+  fi
+fi
+
+# FUTURE implementation :
+# (when our opcua-server code can finally use the vanilla upstream version)
+# Check if libopen62541-iotg fork is available thru related pc file. Otherwise will use default libopen62541.
+# If no opcua lib is found, opcua-server code will not be compiled.
+#no_opcua_lib=0
+#
+#PKG_CHECK_MODULES([open62451],
+#    [open62541-iotg],
+#    [AC_DEFINE([HAVE_OPEN62541_IOTG], [1], [Using open62541 using iotg fork])],
+#    [PKG_CHECK_MODULES([open62541],
+#        [open62541],
+#        [AC_DEFINE([HAVE_OPEN62541_VANILLA], [1], [Using open62541 using upstreamed version])],
+#        [no_opcua_lib=1])])
+#
+#if test "x${no_opcua_lib}" = "x1"; then
+#  AC_MSG_WARN([Both libopen62541 and libopen62541 iot forked are not available in the system. opcua-based test will NOT be compiled.])
+#fi
+#
+#AM_CONDITIONAL([NO_OPCUA_LIB], [test "x$no_opcua" = "x1"])
 
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_FILES([Makefile])