Part 5. Validate the Setup (2024)

This section describes how to run end-to-end traffic from the UE to the edge Core Network.

Start CN5G Core Network

Use the following commands to start the CN5G Core Network.

 
sudo sysctl net.ipv4.conf.all.forwarding=1sudo iptables -P FORWARD ACCEPTcd ~/oai-cn5gdocker compose up -d 

Start theCN5G Edge Application

After the CN5G is started, use the oai-ext-dn container to run IPERF.

 
docker exec -it oai-ext-dn /bin/bash 

Start Aerial cuBB on the gNB

Execute the following command to set up the cuBB container.

 
# Run on host: start a docker terminaldocker exec -it $AERIAL_CUBB_CONTAINER /bin/bash 

Follow the Aerial cuBB documentationto build and run the cuphycontroller. The following instructions are for building and setting up the environmentfor running cuphycontroller. The following commands must be run from inside the cuBB container.

 
cd /opt/nvidia/cuBBexport cuBB_SDK=$(pwd)mkdir build && cd buildcmake ..make -j 

The Aerial cuPHY configuration files are located in the /opt/nvidia/cuBB/cuPHY-CP/cuphycontroller/config directory.For ARC-OTA 1.5, the setup has been validated with cuphycontroller_P5G_FXN.yaml for the Gigabyte server and withcuphycontroller_P5G_FXN_R750.yaml for the Dell R750 server.

Before running the cuphycontroller, edit the cuphycontroller_<xyz>.yaml configuration file to point to thecorrect MAC address of the ORU and the correct PCIe address of the FH interface on the gNB (the following exampleapplies to the Gigabyte server).

 
sed -i "s/ nic:.*/ nic: 0000:b5:00.0/" ${cuBB_SDK}/cuPHY-CP/cuphycontroller/config/cuphycontroller_<xyz>.yamlsed -i "s/ dst_mac_addr:.*/ dst_mac_addr: 6c:ad:ad:00:02:02/" ${cuBB_SDK}/cuPHY-CP/cuphycontroller/config/cuphycontroller_<xyz>.yaml 

When the build is done and the configuration files are updated, exit the container. Run the followingto commit the changes to a new image. The name of the image will be used later in the Docker Composeconfiguration.

 
docker commit $AERIAL_CUBB_CONTAINER cubb-build:24-1 

Creating the NVIPC Source Code Package

In the latest release, NVIPC is no longer included in the OAI repository. You must copy the source package fromthe Aerial cuBB container and add it to the OAI build files. Execute the following to create the nvipc_src.<data>.tar.gzfile.

 
docker exec -it $AERIAL_CUBB_CONTAINER ./cuPHY-CP/gt_common_libs/pack_nvipc.shdocker cp $AERIAL_CUBB_CONTAINER:/opt/nvidia/cuBB/cuPHY-CP/gt_common_libs/nvipc_src.<date>.tar.gz ~/openairinterface5g 

This will create and copy the nvipc_src.<data>.tar.gz archive that is required to build OAI L2+ for Aerial.Instructions can also be found in the Aerial FAPI.

Build gNB Docker Image

In ARC-OTA 1.5, the OAI image is built in two steps. Instructions can be found in theAerial FAPI.

Pre-build Steps for OAI L2

1. When building OAI L2 for Aerial 24-1, remove line 50 and 51 in the docker/Dockerfile.gNB.aerial.ubuntu20 file before building.(ARC-OTA 1.5. includes the OAI 2024.w21 tag and Aerial CUDA-Accelerated RAN (Layer 1) 24-1). This will be mergedto OAI develop branch in the future.

2. In the same file, add moreutils on line 79.

 
moreutils \ 

3. Remove the pinning of p7_thread in the OAI FAPI integration (nfapi/oai_integration/aerial/fapi_vnf_p5.c). To do so,remove lines 59-64 (this change will be merged to the OAI develop branch in the future). Core 8 is occupied by L1 onthe Gigabyte server. On Dell R750, this core is free, but gNB is usually started on numa 0 with odd cores.

   Copy

   Copied!

   See Also

   The Surprising Resilience Of Business Cards In A Digital-First Era | ABC Money

    
   //CPU_SET(8, &cpuset);//s = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), &cpuset);//if (s != 0)// printf("failed to set afinity\n"); 

4. Ensure the the FAPI polling thread is run on an isolated core by modifying line 609 of the nfapi/oai_integration/aerial/fapi_nvIPC.cfile.

   On the Gigabyte server and Dell R750 server, use core 21:

   Copy

   Copied!

    
   stick_this_thread_to_core(21) 

5. There is also a bug regarding PDU length truncation in the OAI 2024.w21 tag used with ARC 1.5. This is already fixed on the develop branch of OAI.The current ARC release has been verified by applying this MRon top of the 2024.w21 OAI 2024.w21 tag.

   Copy

   Copied!

    
   cd ~/openairinterface5g/wget https://gitlab.eurecom.fr/oai/openairinterface5g/-/merge_requests/2797.patchgit apply -3 2797.patch 

Use the below commands to build the OAI L2 code:

 
cd ~/openairinterface5g/docker build . -f docker/Dockerfile.base.ubuntu20 --tag ran-base:latestdocker build . -f docker/Dockerfile.gNB.aerial.ubuntu20 --tag oai-gnb-aerial:latest 

This will create two images:

• ran-base:latest includes the environment to build the OAI source code. This will be used then building the oai-gnb-aerial:latest image.

• oai-gnb-aerial:latest will be used later in the Docker Compose script to create the OAI L2 container.

Running gNB with Docker Compose

ARC-OTA 1.5 includes a Docker Compose configuration for running gNB software. Refer to theOAI instructionson how to run with Docker Compose. The Docker Compose configuration is located inthe following file path:

 
~/openairinterface5g/ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yaml 

Before starting the gNB, you will need to update the docker-compose.yaml file.

1. On line 27, switch the cuBB image to the image that was built, committed, and tagged in the previous steps.

   Copy

   Copied!

    
   image: cubb-build:24-1 

2. On line 30, add sudo to the command.

   Copy

   Copied!

    
   command: bash -c "sudo rm -rf /tmp/phy.log && sudo chmod +x /opt/nvidia/cuBB/aerial_l1_entrypoint.sh && /opt/nvidia/cuBB/aerial_l1_entrypoint.sh" 

3. On line 37, if you follow the guide, change the cpu_set for the OAI container as follows.

   1. Gigabyte server:

      Copy

      Copied!

       
      cpu_set: "13-20" 

   2. Dell R750 server:

      Copy

      Copied!

       
      cpu_set: "23,25,27,29,31,33,35,37" 

   3. SMC-GH server:

      Copy

      Copied!

       
      cpuset: "11-18" 

4. On line 60, add a volume for the oai.log file.

   Copy

   Copied!

    
   - /var/log/aerial:/var/log/aerial 

5. On line 61 (after the volumes), add a command for executing the nr-softmodem. This will replace the default command that is integratedin the docker image. This new command will set the priority and create an oai.log file with time stamp.

   Copy

   Copied!

    
   command: bash -c "chrt -f 99 /opt/oai-gnb/bin/nr-softmodem -O /opt/oai-gnb/etc/gnb.conf | ts | tee /var/log/aerial/oai.log" 

After following the instructions, you should have the following images:

 
$ docker image lsREPOSITORY TAG IMAGE ID CREATED SIZEcubb-build 24-1 be7a5a94f2d3 10 seconds ago 26GBnvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran 24-1-cubb a66cf2a45fad 2 months ago 25.5GBoai-gnb-aerial latest 0856b9969f42 3 weeks ago 4.88GBran-base latest c5d060d23529 3 weeks ago 2.42GB 

Docker Compose will start containers running cuBB and OAI L2+. The Docker Compose script includesan entry-point script for cuBB, which you need to modify before running. The script points atthe cuphycontroller_xxx.yaml configuration that you want to run. This script is located inthe following file path:

 
~/openairinterface5g/ci-scripts/yaml_files/sa_gnb_aerial/aerial_l1_entrypoint.sh 

Before running Docker Compose, update the aerial_l1_entrypoint.sh file.

1. The latest ARC-OTA release does not have to build gdrcopy.

   Copy

   Copied!

    
   # cd "$cuBB_Path"/cuPHY-CP/external/gdrcopy/ || exit 1# ./insmod.sh 

2. In the latest ARC-OTA release, you are not root by default. Add sudo to the following, including P5G_FXN_R750if you are using a Dell R750 or P5G_FXN if you are using .

   Copy

   Copied!

    
   sudo -E "$cuBB_Path"/build/cuPHY-CP/cuphycontroller/examples/cuphycontroller_scf P5G_FXN_R750 

3. Before running the Docker Compose script, you also need to add root for some commands that are run before starting Aerial cuPHY Layer 1.To do so, edit the ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yaml file.

   Copy

   Copied!

    
   command: bash -c " sudo rm -rf /tmp/phy.log && sudo chmod +x /opt/nvidia/cuBB/aerial_l1_entrypoint.sh && /opt/nvidia/cuBB/aerial_l1_entrypoint.sh" 

   Also, OAI L2 must point to the correct configuration by editing the following row in the ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yamlfile. All L2 configurations can be found in targets/PROJECTS/GENERIC-NR-5GC/CONF.

   Copy

   Copied!

    
   - ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/gnb-vnf.sa.band78.273prb.aerial.conf:/opt/oai-gnb/etc/gnb.conf 

4. On line 37 in ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yaml, specify the cores to use for OAI l2.

   1. Gigabyte server:

      Copy

      Copied!

       
      cpuset: "13-20" 

   2. Dell R750 server:

      Copy

      Copied!

       
      cpuset: "23,25,27,29,31,33,35,37" 

   3. SMC-GH server:

      Copy

      Copied!

       
      cpuset: "11-18" 

You can now run ARC-OTA with the below commands.

 
cd ~/openairinterface5g/ci-scripts/yaml_files/sa_gnb_aerialdocker compose up -d// console of cuBBdocker logs -f nv-cubb// console of oaidocker logs -f oai-gnb-aerial// tail -f /var/log/aerial/oai.log 

CUE Connecting to 5G Network

Take the CUE out of airplane mode to start attaching the UE to thenetwork. Make sure that the CUE is in airplane mode before starting OAI L2 stack.

Observe 5G Connect Status

Refer to the Preamble log in the cuphycontroller console output.

Check the Core Network log or CUE log to see whether NAS authentication andPDU session succeeded.

Running E2E IPERF Traffic

Start ping, iperf, or other network app tests after the PDU session connects successfully.

You can install and run the “Magic IPerf” Android application on the CUE for this purpose.

Ping Test

Ping the UE from the CN:

 
docker exec -it oai-ext-dn ping 12.1.1.2 

Ping from the UE to the CN:

 
ping -I 12.1.1.2 192.168.70.135 

Iperf Downlink Test

Perform Iperf downlink test on the UE Side:

 
iperf3 -s 

Perform Iperf downlink test on the CN5G Side:

 
# Test UDP DLdocker exec -it oai-ext-dn iperf3 -u -P 20 -b 25M -t 60 -c 12.1.1.2#Test UDP bidirectionaldocker exec -it oai-ext-dn iperf3 -u --bidir -P 10 -b 50M -t 60 -c 12.1.1.2# Test TCP DLdocker exec -it oai-ext-dn iperf3 -P 4 -b 100M -t 60 -c 12.1.1.2#Test TCP bidirectionaldocker exec -it oai-ext-dn iperf3 --bidir -P 4 -b 100M -t 60 -c 12.1.1.2 

IPERF Uplink Test

Perform Iperf uplink test on the UE Side:

 
iperf3 -s 

Perform Iperf uplink test on the CN5G Side:

 
#UDPdocker exec -it oai-ext-dn iperf3 -u -R -b 120M -t 60 -c 12.1.1.2#TCPdocker exec -it oai-ext-dn iperf3 -R -b 120M -t 60 -c 12.1.1.2 

To stop the containers, use the following commands:

 
docker stop $OAI_GNB_CONTAINERdocker rm $OAI_GNB_CONTAINER