Support ARM in CI (NVIDIA#73)

* Run the test CI stage on both Amd64 and Arm64.
* Document installing QEMU to run CI for Arm64 on an Amd64 system.
* Disable `examples/simple/tests/test_web_query_tool.py` test on Arm due to a known issue with Faiss NVIDIA#72

Requires NVIDIA#59 to be merged first

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: NVIDIA#73

Author: dagardner-nv

.github/workflows/ci_pipe.yml

@@ -80,12 +80,14 @@ jobs:
   test:
     name: Test
     needs: [check]
-    runs-on:  linux-amd64-cpu4
+    runs-on:  linux-${{ matrix.arch }}-cpu4
     timeout-minutes: 30
     container:
       image: ${{ inputs.container }}
     strategy:
       fail-fast: true
+      matrix:
+        arch: ["amd64", "arm64"]
 
     steps:
       - name: Checkout
@@ -94,7 +96,7 @@ jobs:
           fetch-tags: true
           path: 'agentiq'
 
-      - name: Test:linux
+      - name: Test:linux:${{ matrix.arch }}
         shell: bash
         run: ./agentiq/ci/scripts/github/tests.sh
 
@@ -103,7 +105,7 @@ jobs:
         # Upload test results even if the tests fail
         if: ${{ always() }}
         with:
-          name: "test-results"
+          name: "test-results-${{ matrix.arch }}"
           path: "${{ github.workspace }}/tmp/*.xml"
           if-no-files-found: error
 
@@ -152,6 +154,7 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
         with:
+          fetch-depth: 0
           fetch-tags: true
           path: 'agentiq'
 

ci/scripts/run_ci_local.sh

@@ -53,7 +53,7 @@ GIT_COMMIT=$(git log -n 1 --pretty=format:%H)
 # Specifies whether to mount the current git repo or to use a clean clone (the default)
 USE_HOST_GIT=${USE_HOST_GIT:-0}
 
-LOCAL_CI_TMP=${LOCAL_CI_TMP:-${AGENTIQ_ROOT}/.tmp/local_ci_tmp}
+LOCAL_CI_TMP=${LOCAL_CI_TMP:-${AGENTIQ_ROOT}/.tmp/local_ci_tmp/${CI_ARCH}}
 DOCKER_EXTRA_ARGS=${DOCKER_EXTRA_ARGS:-""}
 
 CI_CONTAINER=${CI_CONTAINER:-"ghcr.io/astral-sh/uv:python3.12-bookworm"}

ci/vale/styles/config/vocabularies/aiq/accept.txt

@@ -47,6 +47,7 @@ DB(s?)
 [Ee]ngineerable
 [Ee]val
 [Ee]xplainability
+Faiss
 [Gg]eneratable
 glog
 GPU(s?)

docs/source/advanced/running-ci-locally.md

@@ -57,9 +57,37 @@ From this point you can manually copy/paste the commands which would normally be
 
 | Name | Description | Location |
 |--|--|--|
-| Artifacts | Test results, wheels, and documentation | `.tmp/local_ci_tmp/local_ci_workspace` |
-| Cache | `uv` and `pre-commit` package caches | `.tmp/local_ci_tmp/cache` |
-| Virtual Environment | Python virtual environment | `.tmp/local_ci_tmp/local_ci_workspace/.venv` |
-| Bootstrap Script | The script used to bootstrap the CI environment within the CI container | `.tmp/local_ci_tmp/bootstrap_local_ci.sh` |
+| Artifacts | Test results, wheels, and documentation | `.tmp/local_ci_tmp/<arch>local_ci_workspace` |
+| Cache | `uv` and `pre-commit` package caches | `.tmp/local_ci_tmp/<arch>cache` |
+| Virtual Environment | Python virtual environment | `.tmp/local_ci_tmp/<arch>local_ci_workspace/.venv` |
+| Bootstrap Script | The script used to bootstrap the CI environment within the CI container | `.tmp/local_ci_tmp/<arch>bootstrap_local_ci.sh` |
+
+Where `<arch>` is the architecture of the host machine at time of writing this will be either `amd64` or `arm64`.
 
 > Note: In some situations it may be necessary to delete the `.tmp/local_ci_tmp` directory to clear out old artifacts and caches. This is especially true if you are switching between branches or if you are running into issues with the CI pipeline.
+
+## Multi-Architecture CI
+To run the CI pipeline on a different architecture other than your own, QEMU can be used to emulate the target architecture.
+
+> Note: This assumes you have an amd64 system and want to run the CI pipeline on arm64. If you are using an arm64 and want to emulate amd64, you will need to adjust the commands accordingly.
+
+On an apt based system, this can be done with the following commands:
+```bash
+sudo apt install qemu-utils qemu-system-arm qemu-user-static
+```
+
+Register QEMU with Docker:
+```bash
+docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+```
+
+Verify that the registration was successful:
+```bash
+docker run --platform=linux/arm64 --rm -t ubuntu:noble uname -m
+```
+
+### Run CI on arm64
+The `CI_ARCH` environment variable can be set to the desired architecture to run CI, for example to run the CI pipeline on arm64, you can use the following command:
+```bash
+CI_ARCH=arm64 ./ci/scripts/run_ci_local.sh all
+```

docs/source/release-notes.md

@@ -30,5 +30,6 @@ This is the first general release of AgentIQ.
 - LlamaIndex
 
 ## Known Issues
+- Faiss is currently broken on Arm64. This is a known issue [#72](https://github.com/NVIDIA/AgentIQ/issues/72) caused by an upstream bug in the Faiss library [https://github.com/facebookresearch/faiss/issues/3936](https://github.com/facebookresearch/faiss/issues/3936).
 - AgentIQ applications must use the same name for both the distribution and root package. This is a current implementation limitation and will be addressed in a future release.
 - Refer to [https://github.com/NVIDIA/AgentIQ/issues](https://github.com/NVIDIA/AgentIQ/issues) for an up to date list of current issues.

examples/simple/tests/test_web_query_tool.py

@@ -13,12 +13,17 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import platform
+
+import pytest
 from aiq_simple.register import WebQueryToolConfig
 
 from aiq.builder.workflow_builder import WorkflowBuilder
 from aiq.test.embedder import EmbedderTestConfig
 
 
+@pytest.mark.skipif(platform.machine() == "aarch64",
+                    reason="faiss not working on arm64 https://github.com/NVIDIA/AgentIQ/issues/72")
 async def test_web_query_config():
 
     config = WebQueryToolConfig(webpage_url="https://www.google.com",
@@ -36,6 +41,8 @@ async def test_web_query_config():
         assert fn.description == config.description
 
 
+@pytest.mark.skipif(platform.machine() == "aarch64",
+                    reason="faiss not working on arm64 https://github.com/NVIDIA/AgentIQ/issues/72")
 async def test_web_query_tool():
 
     config = WebQueryToolConfig(webpage_url="https://www.google.com",