Migrating from the Community Edition (VM base) to the Cloud Native edition#

Overview#

This operational guide walks through migration from the community edition, which uses a VM approach, to the cloud native edition, which is primarily a Kubernetes cluster.

Requirements#

Access to the CE VM
Gluu CE version >= 4.5
A Kubernetes cluster, and access to kubectl. You may take a look at the following section to get a better sense on sizing requirements for the Kubernetes cluster.

Migration Steps#

Log in to the server where CE is installed:
```
ssh $USER@$CE_SERVER
```
Back up the data in persistence and save them elsewhere.
Set an environment variable to mark where the root directory of CE installation is.

If using chrooted installation:
```
export CE_HOME=/opt/gluu-server
```
otherwise:
```
export CE_HOME=
```

Prepare manifests files:

Create new directory:
```
mkdir -p $HOME/ce-migration
cd $HOME/ce-migration
```
Take a note about the full path of this directory (for example /root/ce-migration).

If setup.properties.last exists create setup.properties. Otherwise generate setup.properties.:

cp $CE_HOME/install/community-edition-setup/setup.properties.last setup.properties

If setup.properties.last does not exist:

openssl enc -d -aes-256-cbc -in $CE_HOME/install/community-edition-setup/setup.properties.last.enc -out setup.properties

Get all certificates, keys, and keystores:

cp $CE_HOME/etc/certs/*.crt .
cp $CE_HOME/etc/certs/*.key .
cp $CE_HOME/etc/certs/*.pem .
cp $CE_HOME/etc/certs/*.jks .
cp $CE_HOME/etc/certs/*.pkcs12 .
cp $CE_HOME/opt/shibboleth-idp/credentials/*.jks .
cp $CE_HOME/opt/shibboleth-idp/credentials/*.kver .
cp $CE_HOME/opt/shibboleth-idp/conf/datasource.properties .

Get the salt file:
```
cp $CE_HOME/etc/gluu/conf/salt .
```

Get configuration/secret from the persistence that is used with your current CE installation.

LDAP

Run the following LDAP search queries:

$CE_HOME/opt/opendj/bin/ldapsearch \
    --useSSL \
    --trustAll \
    -D "cn=directory manager" \
    -p 1636 \
    -w $LDAP_PASSWD \
    -b "o=gluu" \
    -s sub '(objectClass=gluuConfiguration)' > gluu-configuration.ldif

$CE_HOME/opt/opendj/bin/ldapsearch \
    --useSSL \
    --trustAll \
    -D "cn=directory manager" \
    -p 1636 \
    -w $LDAP_PASSWD \
    -b "o=gluu" \
    -s sub '(objectClass=oxAuthConfiguration)' > oxauth-configuration.ldif

$CE_HOME/opt/opendj/bin/ldapsearch \
    --useSSL \
    --trustAll \
    -D "cn=directory manager" \
    -p 1636 \
    -w $LDAP_PASSWD \
    -b "o=gluu" \
    -s sub '(objectClass=oxAuthClient)' > oxauth-client.ldif

Here's an example of expected .ldif file:

dn: ou=configuration,o=gluu
gluuHostname: 1b4211097aa4
gluuOrgProfileMgt: false
gluuPassportEnabled: false
gluuRadiusEnabled: false
gluuSamlEnabled: false
gluuScimEnabled: false
gluuVdsCacheRefreshEnabled: true

Couchbase

Run the following N1QL queries (in Couchbase UI):

# save the result as gluu-configuration.json manually
SELECT META().id, gluu.*
FROM gluu
WHERE objectClass = 'gluuConfiguration'

# save the result as oxauth-configuration.json manually
SELECT META().id, gluu.*
FROM gluu
WHERE objectClass = 'oxAuthConfiguration'

# save the result as oxauth-client.json manually
SELECT META().id, gluu.*
FROM gluu
WHERE objectClass = 'oxAuthClient'

Here's an example of the expected .json file:

[
    {
        "dn": "ou=configuration,o=gluu",
        "gluuPassportEnabled": false,
        "gluuRadiusEnabled": false,
        "gluuSamlEnabled": false,
        "gluuScimEnabled": false,
        "gluuVdsCacheRefreshEnabled": false,
        "id": "configuration",
        "objectClass": "gluuConfiguration"
    }
]

Spanner

Follow the official docs at https://cloud.google.com/spanner/docs/export (currently the supported format is Avro only).

Here's an example of exported Avro filenames:

gluuConfiguration.avro-00000-of-00001
oxAuthConfiguration.avro-00000-of-00001
oxAuthClient.avro-00000-of-00001

The expected filenames used by config-init container are:

gluu-configuration.avro
oxauth-configuration.avro
oxauth-client.avro

hence you may need to copy them manually from the original Avro files.

SQL

MySQL

Install mysqlsh, then run the following commands:

echo 'select * from gluuConfiguration' | mysqlsh --json=pretty --sql --show-warnings=false --uri=$DBUSER@$DBHOST:$DBPORT/$DBNAME -p > gluu-configuration.json
echo 'select * from oxAuthConfiguration' | mysqlsh --json=pretty --sql --show-warnings=false --uri=$DBUSER@$DBHOST:$DBPORT/$DBNAME -p > oxauth-configuration.json
echo 'select * from oxAuthClient' | mysqlsh --json=pretty --sql --show-warnings=false --uri=$DBUSER@$DBHOST:$DBPORT/$DBNAME -p > oxauth-client.json

Here's an example of the expected .json file:

{
    "hasData": true,
    "rows": [
        {
            "doc_id": "configuration",
            "objectClass": "gluuConfiguration",
            "dn": "ou=configuration,o=gluu",
            "description": null,
            "oxSmtpConfiguration": {
                "v": []
            },
            "gluuVDSenabled": null,
            "ou": "configuration",
            "gluuStatus": null,
            "displayName": null
        }
    ]
}

PostgreSQL

psql -h $DBHOST -p $DBPORT -U $DBUSER -d $DBNAME -W -t -A -o gluu-configuration.json -c 'select json_agg(t) from (select * from "gluuConfiguration") t;'
psql -h $DBHOST -p $DBPORT -U $DBUSER -d $DBNAME -W -t -A -o oxauth-configuration.json -c 'select json_agg(t) from (select * from "oxAuthConfiguration") t;'
psql -h $DBHOST -p $DBPORT -U $DBUSER -d $DBNAME -W -t -A -o oxauth-client.json -c 'select json_agg(t) from (select * from "oxAuthClient") t;'

Here's an example of the expected .json file:

[
    {
        "doc_id": "configuration",
        "objectClass": "gluuConfiguration",
        "dn": "ou=configuration,o=gluu",
        "oxTrustStoreConf": "{\"useJreCertificates\":true}",
        "gluuAdditionalMemory": null,
        "gluuSmtpRequiresAuthentication": null,
        "gluuPassportEnabled": 0,
        "gluuShibFailedAuth": null,
        "gluuAppliancePollingInterval": null,
        "gluuAdditionalBandwidth": null,
        "gluuRadiusEnabled": 0,
        "description": null
    }
]

Log out from the server where CE is installed.

Download manifests files:

scp -r $USER@$CE_SERVER:/root/ce-migration .

Prepare required kubernetes cluster as shown here.

Warning

Do not run helm install before completing next steps below!
Prepare archive and script to extract it:
1. Create archive file contains the downloaded manifests in working directory (assumed $HOME/ce-migration):
```
tar cvzf ce-migration.tar.gz *
```
  and put the archive somewhere else, e.g. file server, that can be downloaded by pod's entrypoint.
2. Create shell script extract-ce-migration.sh to download and extract the archive (the script will be executed by pod's entrypoint)
```
#!/usr/bin/env sh

set -e

# adjust the download URL of the archive
wget -nv http://209.97.172.221:9797/ce-migration.tar.gz -P /tmp
mkdir -p /ce-migration
tar xvf /tmp/ce-migration.tar.gz -C /ce-migration
```
3. Store the script in configmaps:
```
kubectl -n <namespace> create cm extract-ce-migration --from-file=extract-ce-migration.sh
```

Modify the override-values.yaml file from step 8:

global:
  persistence:
    enabled: false
config:
  volumes:
    - name: extract-ce-migration-vol
      configMap:
        name: extract-ce-migration
        defaultMode: 0755
  volumeMounts:
    - name:  extract-ce-migration-vol
      mountPath: /tmp/extract-ce-migration.sh
      subPath: extract-ce-migration.sh
  migration:
    enabled: true
    migrationDir: /ce-migration
    # adjust the data format based on exported persistence used in CE
    migrationDataFormat: ldif

Persistence type	Migration Data format	File extension
LDAP	`ldif`	`.ldif`
Couchbase	`couchbase+json`	`.json`
MySQL	`mysql+json`	`.json`
PostgreSQL	`postgresql+json`	`.json`
Spanner	`spanner+avro`	`.avro`

Install the chart (refer to guide from step 8):

helm install <release-name> gluu/gluu --version <version> -f override-values.yaml -n <namespace>

Import backup data from step 2 into the persistence manually.

Restart main services, for examples:

kubectl rollout restart deployment <release-name>-auth-server -n <namespace>
kubectl rollout restart statefulset <release-name>-oxtrust -n <namespace>

If new additional services were deployed that originally were not on the source CE VM (i.e. SCIM, Fido2, etc), the persistence job must be enabled to fill the missing entries (existing entries will not be modified)

Note

Some configuration may need to be modified manually via oxTrust UI.
1. Modify override-values.yaml:
```
global:
  persistence:
    enabled: true
  upgrade:
    enabled: true
config:
  migration:
    enabled: false
```
2. Run helm upgrade:
```
helm upgrade <release-name> gluu/gluu --version <version> -f override-values.yaml -n <namespace>
```