Skip to content

Cred API Commercial Review Environment Setup

This document outlines the steps required to set up the Cred API Commercial Review environment.

1. Introduction

Cred API commercial review environment allows you to build and test your environment, with isolated components.

Review resources are deployed on GKE Autopilot, with Cloud Run managing the GKE cluster.

For each review app we create a separate namespace featuring a standalone Redis instance deployed alongside a cred-api-commercial app with env variables.

Additionally, an optional, separate database is available in case the environment requires one. In all other cases review apps are using review-database.

The database is initialized from a development backup and is updated every Monday to ensure it remains current with the development environment.

2. Create Review App

In order to create a review app you just need to create a PR in repo cred-api-commercial. After PR is created you should see GitHub Action running.

GitHub Action Running

Upon completion, the GitHub action will post a message to the PR indicating either success or an error.

GitHub Action Success

The URL may not be available immediately, as GCP still provisioning it in the background but it becomes available in a minute or so.

  • GCP Secrets URL: where you can edit and add secrets for environment, for now after secrets changed you need to redeploy application manually (but automation is coming)
  • Review App Logs URL: is where you can check logs for the application

2.1 Create a Review App with DB

In order to create a review app with DB you need to create a PR in repo cred-api-commercial and add label database, and create PR.

Database Label

Upon completion, the GitHub action will post a message to the PR indicating either success or an error.

2.2 Connect to DB

Prerequisites

Before you begin, ensure you have the following:

  • gcloud and active GCP account in cred-review project

To be able to run commands make sure beforehand you run:

gcloud auth login
gcloud config set project cred-review

Connection

All dbs using same credentials as dev db

For connection to master db that all the review app environments use by default you can run this command in your local:

gcloud container clusters get-credentials review-app-dev --region us-central1 --project cred-review \
  && kubectl port-forward --namespace review-dbs $(kubectl get pod --namespace review-dbs --selector="app=postgres-master" --output jsonpath='{.items.metadata.name}') 8080:5432

To connect to database created separately for your application, follow these steps:

Open:

https://console.cloud.google.com/kubernetes/discovery?inv=1\&invt=Ab21vQ\&organizationId=252201914815\&project=cred-review

Find the DB that was created for your review app it should be with following naming: postgres-{branch name}

Open it and select Port Forwarding:

Database Port Forwarding

You can change port 8080 to any port you would like to connect on your local computer.

After running port forwarding command in your terminal set up connection to your db in IDE of your choice set Host to localhost and port to whatever port it's forwarding to and add credentials:

Database Connection

2.3 Connect to Redis

In order to connect to Redis open this url select your namespace (same as branch name) you should see redis-service-{branch-name} open it and select Port Forwarding and copy paste command into your PC terminal, after that use your IDE tool and connect to redis using:

  • Host: localhost
  • Port: check in port forwarding command
  • Password: should be your redis password which you can check in GCP secrets REDISCLOUD_URL

2.4 Change/add Secrets

To be able to find secrets, open this url locate your review app secret, which should be named master-secret-{branch-name}.

Select the secret, retrieve its latest value, and then choose "View Secret Value."

Secrets Manager

Copy the value to your text editor and make the necessary changes. Then, navigate to the Secret page and add a New Version. Tick "Disable all past versions" and paste the value into "Secret Value." Finally, add the New Version.

Update Secret

After new version is added both api and worker will be redeployed.

3. Recreate Environment

In order to recreate your env you just need to re-run your branch github action for that open Github Action Review App management page:

https://github.com/credinvest/cred-api-commercial/actions/workflows/create-review-app.yml

Locate your branch and select Re-run all jobs:

Re-run Jobs

4. Execute Task

In order to run command open workload webpage and find your app review namespace and select application or worker:

Workload Selection

Run in your terminal or cloud shell replacing NAMESPACE, MANAGED_PODS_NAME, and YOUR_TASK_HERE:

kubectl exec -it MANAGED_PODS_NAME_WORKER -n NAMESPACE -- YOUR_TASK_HERE

Example:

kubectl exec -it commercial-api-com-24311-main-contact-em-worker-644cfd8fc6pfdlp -n com-24311-main-contact-em -- node dist/worker/tasks/execute-task.js --t CREATE_CUSTOM_FIELDS_FROM_TEMPLATE

5. Environment Manual Invocation

(Skip Manual Invocation unless necessary)

5.1 Prerequisites

Before you begin, ensure you have the following:

  • gcloud and active GCP account
  • curl or http

To be able to run commands make sure beforehand you run:

gcloud auth login

5.2 Environment Creation and Management

5.2.1 Environment Creation (without DB)

To create an environment make sure you run this command and provide a valid branch name.

If the environment already exists and you run the command it will be recreated from scratch.

curl -X POST -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://cluster-management-398157627861.us-central1.run.app/environments/create/branch_name

When you run this command cloud run follows this logic you can also see what's going on in cloud run logs here (in the future will be streaming logs to terminal):

  1. Cloud Build Process: The build process typically takes 7-8 minutes.
  2. Cloud build Trigger: A trigger has been set up in Cloud Build to automatically build and deploy a new image whenever new commits are made.
  3. Artifact Registry & Image Tagging: Once the build is complete, the image is pushed to the commercial-api-review Artifact Registry and tagged with the corresponding branch name.
  4. Namespace Creation: A namespace is created using the branch name.
  5. Secret Management: Master Secrets (containing all necessary secrets) are copied to the newly created namespace.
  6. Redis Instance Creation: A Redis instance is provisioned within the namespace.
  7. Application Deployment: The application is then deployed in the same namespace, utilizing the image created in step 2.
  8. Load Balancer & DNS: If the API initiates successfully and maintains a healthy status, a Load Balancer is established, and a DNS name is assigned in the format https://[branch_name].k8s-dev.credplatform.com.

5.2.2 Environment Creation with DB (this one is taking longer)

curl -X POST -H "Authorization: Bearer $(gcloud auth print-identity-token)" "https://cluster-management-398157627861.us-central1.run.app/environments/create/branch_name?create_db=true"

The difference between the previous is adding flag create_db=true all logs are here.

If the environment already exists and you run the command it will be recreated from scratch.

  1. Cloud Build Process: The build process typically takes 7-8 minutes.
  2. Cloud build Trigger: A trigger has been set up in Cloud Build to automatically build and deploy a new image whenever new commits are made.
  3. Artifact Registry & Image Tagging: Once the build is complete, the image is pushed to the commercial-api-review Artifact Registry and tagged with the corresponding branch name.
  4. Namespace Creation: A namespace is created using the branch name.
  5. Database is Created from restored Snapshot of dev-commercial
  6. Secret Management: Master Secrets (containing all necessary secrets) are copied to the newly created namespace.
  7. Redis Instance Creation: A Redis instance is provisioned within the namespace.
  8. Application Deployment: The application is then deployed in the same namespace, utilizing the image created in step 2.
  9. Load Balancer & DNS: If the API initiates successfully and maintains a healthy status, a Load Balancer is established, and a DNS name is assigned in the format https://[branch_name].k8s-dev.credplatform.com.

5.2.3 Delete Environment

To delete an environment make sure you run this command and provide valid branch name:

curl -X DELETE -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://cluster-management-398157627861.us-central1.run.app/environments/branch_name

This removes all the created resources like:

  • Cloud Build Trigger
  • Artifact Registry & Image Tagging
  • Namespace Creation
  • Secret Management
  • Redis Instance Creation
  • Application Deployment
  • Load Balancer & DNS

6. Checking Workloads and Status

To check the workload, follow these steps:

Navigate to the Workloads page to view running pods. Select the appropriate namespace (corresponding to the branch name) to see deployed resources, their status, or logs.

Connecting to Redis or PostgreSQL

Follow these steps to connect to Redis or PostgreSQL:

  1. Navigate to the Workload page and find the Redis or PostgreSQL service.
  2. Click on the service name to view its details.
  3. Scroll to the Port Forwarding section at the bottom of the page.
  4. Copy the provided kubectl command to forward traffic from your localhost to the service.
  5. Run it on your computer

If you want to connect to main review psql you can find it on this page the command you need to run on your computer to get traffic forwarder is this one:

gcloud container clusters get-credentials review-app-dev --region us-central1 --project cred-review \
  && kubectl port-forward --namespace review-dbs $(kubectl get pod --namespace review-dbs --selector="app=postgres-master" --output jsonpath='{.items.metadata.name}') 8080:5432

You can change port 8080 to any port you would like to connect on your local computer.