Skip to content
This repository was archived by the owner on Jan 30, 2025. It is now read-only.

Commit 31e8d0a

Browse files
authored
Correct wiring of the subscriptions-ci pipeline and prompt for NVA firewall username & password (Azure#285)
1 parent 229b144 commit 31e8d0a

File tree

9 files changed

+46
-34
lines changed

9 files changed

+46
-34
lines changed

.pipelines/README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,16 @@ The following top-level pipelines are present in the `.pipelines/` repository fo
1818
| 2 | Platform Logging | `platform-logging.yml` | platform-logging-ci
1919
| 3 | Policy | `policy.yml` | policy-ci
2020
| 4 | Roles | `roles.yml` | roles-ci
21-
| 5 | Networking | `platform-connectivity-hub-nva.yml` | platform-connectivity-hub-nva-ci
22-
| 6 | Subscription | `subscription.yml` | subscription-ci
21+
| 5a | Networking (NVA) | `platform-connectivity-hub-nva.yml` | platform-connectivity-hub-nva-ci
22+
| 5b | Networking (Azure Firewall) | `platform-connectivity-hub-azfw-policy.yml` | platform-connectivity-hub-azfw-policy-ci
23+
| 5b | Networking (Azure Firewall) | `platform-connectivity-hub-azfw.yml` | platform-connectivity-hub-azfw-ci
24+
| 6 | Subscriptions | `subscriptions.yml` | subscriptions-ci
2325

2426
These pipelines need to be run in the order specified. For example, the `Policy` pipeline is dependent on resources deployed by the `Platform Logging` pipeline. Think of it as a layered approach; once the layer is deployed, it only requires re-running if some configuration at that layer changes.
2527

26-
In the default implementation, the `Management Groups`, `Platform Logging`, `Policy`, and `Roles` pipelines are run automatically (trigger) whenever a related code change is detected on the `main` branch. The `Networking` and `Subscription` pipelines do not run automatically (no trigger). This behavior can be changed by modifying the corresponding YAML pipeline definition files.
28+
There are two distinct `Networking` pipelines, each deploys the hub side of a hub & spoke network topology. The `Networking (NVA)` option is intended for environments with a Network Virtual Appliance, and the `Networking (Azure Firewall)` option is intended for environments using Azure Firewall.
29+
30+
In the default implementation, the `Management Groups`, `Platform Logging`, `Policy`, and `Roles` pipelines are run automatically (trigger) whenever a related code change is detected on the `main` branch. The `Networking` and `Subscriptions` pipelines do not run automatically (no trigger). This behavior can be changed by modifying the corresponding YAML pipeline definition files.
2731

2832
In the default implementation, the `Roles` and `Platform Logging` pipelines are run automatically after a successful run of the `Management Groups` pipeline, and the `Policy` pipeline is run automatically after a successful run of the `Platform Logging` pipeline. Again, this behavior can be changed by modifying the corresponding YAML pipeline definition files.
2933

docs/archetypes/authoring-guide.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -116,12 +116,12 @@ Each archetype is intended to be self-contained and provides all deployment temp
116116
117117
6. Create a JSON Schema definition for the archetype. Consider using a tool such as [JSON to Jsonschema](https://jsonformatter.org/json-to-jsonschema) to generate the initial schema definition that you customize. For all common features, you must reference the existing definitions for the types. See example: [schemas/latest/landingzones/lz-generic-subscription.json](../../schemas/latest/landingzones/lz-generic-subscription.json)
118118
119-
7. Verify archetype deployment through `subscription-ci` Azure DevOps Pipeline. More information on the pipeline can be found in [Azure DevOps Onboarding Guide](../onboarding/ado.md#step-8--configure-subscription-archetypes).
119+
7. Verify archetype deployment through `subscriptions-ci` Azure DevOps Pipeline. More information on the pipeline can be found in [Azure DevOps Onboarding Guide](../onboarding/ado.md#step-8--configure-subscription-archetypes).
120120
121121
- Create a subscription JSON Parameters file per [deployment instructions](#deployment-instructions).
122122
- Run the pipeline by providing the subscription guid
123123
124-
`subscription-ci` pipeline will automatically identify the archetype, the subscription and region based on the file name. The JSON Schema is located by the archetype name and used for pre-deployment verification.
124+
`subscriptions-ci` pipeline will automatically identify the archetype, the subscription and region based on the file name. The JSON Schema is located by the archetype name and used for pre-deployment verification.
125125
126126
Once verifications are complete, the pipeline will move the subscription to the target management group (based on the folder structure) and execute `main.bicep`.
127127
@@ -175,7 +175,7 @@ An archetype can deploy & configure any number of Azure services. For consisten
175175
- **Subscription Tags** - configures subscription tags
176176
- **Resource Tags** - configures tags on resource groups
177177
178-
> **Log Analytics Workspace integration**: `main.bicep` must accept an input parameter named `logAnalyticsWorkspaceResourceId`. This parameter is automatically set by `subscription-ci` Pipeline based on the environment configuration. This parameter is used to link Microsoft Defender for Cloud to Log Analytics Workspace.
178+
> **Log Analytics Workspace integration**: `main.bicep` must accept an input parameter named `logAnalyticsWorkspaceResourceId`. This parameter is automatically set by `subscriptions-ci` Pipeline based on the environment configuration. This parameter is used to link Microsoft Defender for Cloud to Log Analytics Workspace.
179179
180180
Input parameters for common features are:
181181
@@ -276,7 +276,7 @@ As a result, we could either
276276

277277
- have Azure deploy the archetype and fail on invalid inputs. An administrator would have to deploy multiple times to fix all errors; or
278278

279-
- attempt to detect invalid inputs as a pre-check in our `subscription-ci` pipeline.
279+
- attempt to detect invalid inputs as a pre-check in our `subscriptions-ci` pipeline.
280280

281281
We chose to check the input parameters prior to deployment to identify misconfigurations faster. Validations are performed using JSON Schema definitions. These definitions are located in [schemas/latest/landingzones](../../schemas/latest/landingzones) folder.
282282

@@ -341,7 +341,7 @@ These parameter files are located in [config/subscription](../../config/subscrip
341341

342342
Immediate subfolder defines the environment which is based on Azure DevOps Organization (i.e. `CanadaESLZ`) & Git branch name (i.e. `main`), for example the subfolder will be called `CanadaESLZ-main`. You can have many environments based on Git branch names such as `CanadaESLZ-feature-1`, `CanadaESLZ-dev`, etc.
343343

344-
ARM parameter files are used by `subscription-ci` Azure DevOps Pipeline when configuring subscriptions with Azure resources. The pipeline will detect environment, management group, subscription, deployment location and deployment parameters using the folder hierarchy, file name and file content.
344+
ARM parameter files are used by `subscriptions-ci` Azure DevOps Pipeline when configuring subscriptions with Azure resources. The pipeline will detect environment, management group, subscription, deployment location and deployment parameters using the folder hierarchy, file name and file content.
345345

346346
For example when the file path is:
347347

@@ -395,7 +395,7 @@ There are two approaches for achieving uniquness:
395395
396396
In this approach, you must ensure all management group ids are unique yourself.
397397
398-
The `subscription-ci` management group detection logic is built to accommodate both scenarios.
398+
The `subscriptions-ci` management group detection logic is built to accommodate both scenarios.
399399
400400
**To support approach #1:**
401401
@@ -408,7 +408,7 @@ The `subscription-ci` management group detection logic is built to accommodate b
408408
- DevTest
409409
```
410410
411-
- `subscription-ci` will then take the folder structure and concatenate it to create the management group id. In this example `DevTest` management group id will be `pubsecLandingZonesDevTest`.
411+
- `subscriptions-ci` will then take the folder structure and concatenate it to create the management group id. In this example `DevTest` management group id will be `pubsecLandingZonesDevTest`.
412412
413413
**To support approach #2:**
414414
@@ -421,4 +421,4 @@ The `subscription-ci` management group detection logic is built to accommodate b
421421
- DevTest
422422
```
423423
424-
- `subscription-ci` will then take the folder name as the structure (since there aren't any sub folders). In this example `DevTest` management group id will be `DevTest`.
424+
- `subscriptions-ci` will then take the folder name as the structure (since there aren't any sub folders). In this example `DevTest` management group id will be `DevTest`.

docs/architecture.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,7 @@ Use the [Azure DevOps Pipelines](onboarding/azure-devops-pipelines.md) onboardin
556556
| Platform – Hub Networking using NVAs | platform-connectivity-hub-nva.yml | platform-connectivity-hub-nva-ci | Configures Hub Networking with Fortigate Firewalls. | spn-azure-platform-ops | None |
557557
| Platform – Hub Networking with Azure Firewall - Firewall Policy | platform-connectivity-hub-azfw-policy.yml | platform-connectivity-hub-azfw-policy-ci | Configures Azure Firewall Policy. A policy contains firewall rules and firewall configuration such as enabling DNS Proxy. Firewall policies can be updated independently of Azure Firewall. | spn-azure-platform-ops | None |
558558
| Platform – Hub Networking with Azure Firewall | platform-connectivity-hub-azfw.yml | platform-connectivity-hub-azfw-ci | Configures Hub Networking with Azure Firewall. | spn-azure-platform-ops | None |
559-
| Subscriptions | subscription.yml | subscription-ci | Configures a new subscription based on the archetype defined in the configuration file name. | spn-azure-platform-ops | None |
559+
| Subscriptions | subscriptions.yml | subscriptions-ci | Configures a new subscription based on the archetype defined in the configuration file name. | spn-azure-platform-ops | None |
560560
| Pull Request Validation | pull-request-check.yml | pull-request-validation-ci | Checks for breaking changes to Bicep templates & parameter schemas prior to merging the change to main branch. This pipeline must be configured as a check for the `main` branch. | spn-azure-platform-ops | None |
561561

562562
### 9.4 Release Process

docs/onboarding/azure-devops-pipelines.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1500,7 +1500,7 @@ In order to configure audit stream for Azure Monitor, identify the following inf
15001500
5. Select Existing Azure Pipeline YAML file
15011501
6. Identify the pipeline in `.pipelines/subscriptions.yml`.
15021502
7. Save the pipeline (don't run it yet)
1503-
8. Rename the pipeline to `subscription-ci`
1503+
8. Rename the pipeline to `subscriptions-ci`
15041504

15051505
2. Create a subscription configuration file (JSON)
15061506

@@ -1525,9 +1525,9 @@ In order to configure audit stream for Azure Monitor, identify the following inf
15251525
3. Run the subscription pipeline
15261526

15271527
1. In Azure DevOps, go to Pipelines
1528-
2. Select the `subscription-ci` pipeline and run it.
1528+
2. Select the `subscriptions-ci` pipeline and run it.
15291529

1530-
> The `subscription-ci` pipeline YAML is configured, by default, to **not** run automatically; you can change this if desired.
1530+
> The `subscriptions-ci` pipeline YAML is configured, by default, to **not** run automatically; you can change this if desired.
15311531

15321532
3. In the Run Pipelines dialog window, enter the first 4 digits of your new subscription configuration file name (4 is usually enough of the GUID to uniquely identify the subscription) between the square brackets in the `subscriptions` parameter field. For example: `[802e]`.
15331533

docs/onboarding/azure-devops-scripts.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,6 @@ Next, edit the newly created file, using the guidance in the following table.
227227
| DEVOPS_SE_NAME | Azure DevOps service endpoint name. | spn-azure-platform-ops
228228
| DEVOPS_SE_TEMPLATE | File name for the generated Azure DevOps service endpoint template JSON file. | service-endpoint.AzDevOpsOrg.json
229229
| DEVOPS_VARIABLES_GROUP_NAME | Azure DevOps variable group name. Leave this set to `firewall-secrets` as the YAML pipeline for networking is hard-coded to use this value. | firewall-secrets
230-
| DEVOPS_VARIABLES_VALUES | Specify values for the NVA firewall username and password in format `key=value key=value`. Replace `YourUsername` and `YourPassword` in the example with your values. DO NOT commit changes that include username and password plaintext values to your repository. | var-hubnetwork-nva-fwUsername=YourUserName var-hubnetwork-nva-fwPassword=YourPassword
231230
| DEVOPS_VARIABLES_ARE_SECRET | Indicates whether variables in the variable group are marked as secret. Possible values are `true` or `false`. Recommend using `true` unless you plan to reconfigure your variable group to use another secure source such as KeyVault. | true
232231
| DEVOPS_OUTPUT_DIR | Name of temporary folder for generated files. | .\output
233232

@@ -261,7 +260,7 @@ Run the `create-pipelines.bat` script to create the landing zone pipelines:
261260
- platform-connectivity-hub-nva-ci
262261
- platform-connectivity-hub-azfw-ci
263262
- platform-connectivity-hub-azfw-policy-ci
264-
- subscription-ci
263+
- subscriptions-ci
265264

266265
If you would rather perform these steps manually, detailed guidance is available in the following sections of the [Azure DevOps Pipelines Onboarding Guide](./azure-devops-pipelines.md):
267266

@@ -296,7 +295,7 @@ Detailed guidance on these configuration requirements is available in the [Azure
296295

297296
### Run pipelines
298297

299-
Run the `run-pipelines.bat` script to interactively run individual landing zone pipelines. Note that at present time the `subscription-ci` pipeline is not included in the list of runnable pipelines as the script requires additional work to enable that capability.
298+
Run the `run-pipelines.bat` script to interactively run individual landing zone pipelines. Note that at present time the `subscriptions-ci` pipeline is not included in the list of runnable pipelines as the script requires additional work to enable that capability.
300299

301300
### Clear environment variables used by scripts
302301

scripts/onboarding/create-pipelines.bat

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ choice /C YN /M "Do you want to proceed?"
2222
if errorlevel 2 exit /b 0
2323

2424
REM Process all pipeline definitions
25-
for %%N in (management-groups roles platform-logging policy platform-connectivity-hub-nva platform-connectivity-hub-azfw platform-connectivity-hub-azfw-policy subscription) do (
25+
for %%N in (management-groups roles platform-logging policy platform-connectivity-hub-nva platform-connectivity-hub-azfw platform-connectivity-hub-azfw-policy subscriptions) do (
2626

2727
REM Check for pipeline existence
2828
set FOUND=

scripts/onboarding/create-variable-group.bat

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ echo.
1414
echo DevOps Organization: %DEVOPS_ORG%
1515
echo DevOps Project: %DEVOPS_PROJECT_NAME%
1616
echo DevOps Variable Group: %DEVOPS_VARIABLES_GROUP_NAME%
17-
echo DevOps Variables: %DEVOPS_VARIABLES_VALUES%
1817
echo DevOps Variables are Secret: %DEVOPS_VARIABLES_ARE_SECRET%
1918
echo.
2019
choice /C YN /M "Do you want to proceed?"
@@ -33,8 +32,18 @@ if defined ID (
3332
)
3433

3534
REM Create the variable group
36-
echo Creating variable group [%DEVOPS_VARIABLES_GROUP_NAME%] with variables: %DEVOPS_VARIABLES_VALUES%...
37-
call az pipelines variable-group create --name %DEVOPS_VARIABLES_GROUP_NAME% --authorize true --query "[?name=='%DEVOPS_VARIABLES_GROUP_NAME%'].id | [0]" -o tsv --org %DEVOPS_ORG% --project %DEVOPS_PROJECT_NAME% --variables %DEVOPS_VARIABLES_VALUES%
35+
echo Enter NVA username and password to set variables in DevOps variable group [%DEVOPS_VARIABLES_GROUP_NAME%]
36+
echo.
37+
echo **********************************************************************
38+
echo CAUTION: your input is not masked, i.e. it will be visible on-screen
39+
echo **********************************************************************
40+
echo.
41+
set /P NVA_USERNAME=Enter the user name for the NVA firewall:
42+
set /P NVA_PASSWORD=Enter the password for the NVA firewall:
43+
echo.
44+
45+
echo Creating variable group [%DEVOPS_VARIABLES_GROUP_NAME%]...
46+
call az pipelines variable-group create --name %DEVOPS_VARIABLES_GROUP_NAME% --authorize true --query "[?name=='%DEVOPS_VARIABLES_GROUP_NAME%'].id | [0]" -o tsv --org %DEVOPS_ORG% --project %DEVOPS_PROJECT_NAME% --variables var-hubnetwork-nva-fwUsername=%NVA_USERNAME% var-hubnetwork-nva-fwPassword=%NVA_PASSWORD%
3847
echo.
3948
echo Variable group [%DEVOPS_VARIABLES_GROUP_NAME%] has been created.
4049
echo.
@@ -43,9 +52,17 @@ echo.
4352
echo RECOMMENDED that you use the Azure DevOps portal to restrict access to this
4453
echo variable group to only the `platform-connectivity-hub-nva` pipeline.
4554
echo.
46-
echo RECOMMENDED that you DO NOT commit to your repository any changes made
47-
echo to this file that include a plaintext username or password.
48-
echo.
4955

5056
REM Set variables as secret in Azure DevOps if requested
51-
if "%DEVOPS_VARIABLES_ARE_SECRET%" == "true" call update-variable-group.bat true
57+
if "%DEVOPS_VARIABLES_ARE_SECRET%" == "true" (
58+
echo.
59+
echo Setting variables in Azure DevOps variable group [%DEVOPS_VARIABLES_GROUP_NAME%] as secret...
60+
echo.
61+
call update-variable-group.bat true
62+
) else (
63+
echo.
64+
echo **************************************************************************
65+
echo WARNING: NVA firewall variables are not marked as secret in Azure DevOps
66+
echo **************************************************************************
67+
echo.
68+
)

scripts/onboarding/set-variables.DevOpsOrgName.bat

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,10 +47,6 @@ set DEVOPS_SE_TEMPLATE=service-endpoint.DEVOPS-ORG-NAME.json
4747
REM Do not change this value (hard-coded in YAML pipeline definition)
4848
set DEVOPS_VARIABLES_GROUP_NAME=firewall-secrets
4949

50-
REM Variables is a space-delimited key=value string. Provide values for
51-
REM 'var-hubnetwork-nva-fwUsername' and 'var-hubnetwork-nva-fwPassword'.
52-
set DEVOPS_VARIABLES_VALUES=var-hubnetwork-nva-fwUsername=YourUserName var-hubnetwork-nva-fwPassword=YourPassword
53-
5450
REM Are variables in the firewall-secrets group marked as secret? 'true' or 'false'.
5551
set DEVOPS_VARIABLES_ARE_SECRET=true
5652

scripts/onboarding/set-variables.ocag148outlook.bat

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,10 +47,6 @@ set DEVOPS_SE_TEMPLATE=service-endpoint.ocag148outlook.json
4747
REM Do not change this value (hard-coded in YAML pipeline definition)
4848
set DEVOPS_VARIABLES_GROUP_NAME=firewall-secrets
4949

50-
REM Variables is a space-delimited key=value string. Provide values for
51-
REM 'var-hubnetwork-nva-fwUsername' and 'var-hubnetwork-nva-fwPassword'.
52-
set DEVOPS_VARIABLES_VALUES=var-hubnetwork-nva-fwUsername=YourUserName var-hubnetwork-nva-fwPassword=YourPassword
53-
5450
REM Are variables in the firewall-secrets group marked as secret? 'true' or 'false'.
5551
set DEVOPS_VARIABLES_ARE_SECRET=true
5652

0 commit comments

Comments
 (0)