GoldenGate 23ai Multi-Deployment with Configuration Service
This is the second post in Oracle GoldenGate 23ai (OGG 23ai) Configuration Service series. Before reading further, I highly recommend reading the first article that discusses what is Configuration Service and the requirements for setting it up.
In this article, I will discuss how to set up a second OGG 23ai deployment with existing service manager and existing DB backend for Configuration Service.
Consider the following architecture.
Env. Details -
OGG 23ai for Oracle and PostgreSQL(PG) on same VM host.
In the previous article I discussed how Configuration Service stores configuration data inside a database table. Here, I will describe the process of creating a second deployment for OGG 23ai PG with existing Service Manager from OGG 23ai Oracle deployment!
Read the last statement carefully. OGG Service Manager was created as part of OGG 23ai Oracle deployment. This Service Manager will also manage the OGG 23ai PG deployment. Service Manager starts the Configuration Service on deployment startup and is aware of backend details. Hence, they are not required during the OGG PG deployment creation.
Let’s first look at the OGG 23ai PG deployment creation process followed by the process hierarchy. (Pardon the potato quality screengrabs.)
As seen in the above image, installer automatically detects Service Manager listening on port 7500. So, there is a choice for selecting existing Service Manager or creating a new Service Manager on step 1.
Step 2 — Provide credentials for Service Manager Administrator account.
Step 3 — Provide OGG 23ai PG deployment values such as deployment home, ports for administration, distribution, receiver and performance metrics service. Optionally provide StatsD server and port details ( more on this in another post :-) )
Step 4 — In this penultimate step, provide OGG 23ai PG administrator username and password.
The next two screens are just summary screens confirming values provided and options to create a response file. The entire process took a few minutes on my one OCPU machine hosting Oracle 23ai Database Free, PostgreSQL 16, MySQL 8, OGG 23ai Oracle+PG and lots more. It is blazing fast.
If you notice I did not provide Configuration Service values at any stage. So how does the OGG 23ai PG deployment know which database to connect for retrieving and storing data?
Consider following scenario, OracleGoldenGate service is started via following command.
[root@gg23test ~]# systemctl start OracleGoldenGate
[root@gg23test ~]# systemctl status OracleGoldenGate -l
● OracleGoldenGate.service - Oracle GoldenGate Service Manager
Loaded: loaded (/etc/systemd/system/OracleGoldenGate.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2024-05-22 06:16:28 GMT; 11s ago
Main PID: 609015 (ServiceManager)
Tasks: 30 (limit: 202672)
Memory: 337.1M
CGroup: /system.slice/OracleGoldenGate.service
├─ 6866 /dbgg/oggsw/oggorasw/bin/ConfigService -f /dbgg/oggservdep/oggoraservdep/var/lib/conf/ConfigService/Confi>
└─609015 /dbgg/oggsw/oggorasw/bin/ServiceManager --inventory '/dbgg/oggservdep/oggoraservdep/etc/conf'
May 22 06:16:28 gg23test systemd[1]: This usually indicates unclean termination of a previous run, or service implementation >
May 22 06:16:28 gg23test systemd[1]: Started Oracle GoldenGate Service Manager.
May 22 06:16:29 gg23test ServiceManager[609015]: 2024-05-22T06:16:29.983+0000 INFO | Configuring user authorization enabled.
May 22 06:16:29 gg23test ServiceManager[609015]: 2024-05-22T06:16:29.992+0000 INFO | Configuring user authorization secure st>
May 22 06:16:29 gg23test ServiceManager[609015]: 2024-05-22T06:16:29.992+0000 INFO | Configuring user session simple store pa>
May 22 06:16:30 gg23test OracleGoldenGate.sh[609015]: Oracle GoldenGate Service Manager for Oracle
May 22 06:16:30 gg23test OracleGoldenGate.sh[609015]: Version 23.4.0.24.05 OGGCORE_23.0.0.0.0_LINUX.X64_240426.1726_FBO
May 22 06:16:30 gg23test OracleGoldenGate.sh[609015]: Copyright (C) 1995, 2024, Oracle and/or its affiliates. All rights rese>
May 22 06:16:30 gg23test OracleGoldenGate.sh[609015]: Linux, x64, 64bit (optimized) on Apr 26 2024 21:52:50
May 22 06:16:30 gg23test OracleGoldenGate.sh[609015]: Operating system character set identified as UTF-8.Notice the CGroup( control group) values in output above. systemctl starts the ServiceManager which spawns the Configuration Service process. We can confirm the same by looking at currently executing process via ‘ps’ command.
[root@gg23test ~]# ps -ef|grep oggsw
oracle 6866 1 0 May20 ? 00:02:46 /dbgg/oggsw/oggorasw/bin/ConfigService -f /dbgg/oggservdep/oggoraservdep/var/lib/conf/ConfigService/ConfigService.json
oracle 609015 1 7 06:16 ? 00:02:20 /dbgg/oggsw/oggorasw/bin/ServiceManager --inventory '/dbgg/oggservdep/oggoraservdep/etc/conf'
root 612062 608437 0 06:46 pts/1 00:00:00 grep --color=auto oggsw
[root@gg23test ~]#At this stage both deployments are not started.
We can confirm from the shiny new OGG 23ai GUI that Service Manager is currently up (because we can login, duh!), and both the deployments are stopped.
I will start Oracle deployment ‘ogg23oracle’ and review the processes that are started as part of the deployment startup sequence.
[root@gg23test ~]# ps -ef|grep oggsw
oracle 6866 1 0 May20 ? 00:03:19 /dbgg/oggsw/oggorasw/bin/ConfigService -f /dbgg/oggservdep/oggoraservdep/var/lib/conf/ConfigService/ConfigService.json
oracle 609015 1 8 06:16 ? 00:03:18 /dbgg/oggsw/oggorasw/bin/ServiceManager --inventory '/dbgg/oggservdep/oggoraservdep/etc/conf'
oracle 613865 609015 5 06:56 ? 00:00:05 /dbgg/oggsw/oggorasw/bin/adminsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-adminsrvr-config.dat
oracle 613910 609015 1 06:56 ? 00:00:01 /dbgg/oggsw/oggorasw/bin/distsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-distsrvr-config.dat
oracle 613982 609015 0 06:56 ? 00:00:00 /dbgg/oggsw/oggorasw/bin/pmsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-pmsrvr-config.dat
oracle 614038 609015 0 06:56 ? 00:00:00 /dbgg/oggsw/oggorasw/bin/recvsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-recvsrvr-config.dat
root 614199 608437 0 06:57 pts/1 00:00:00 grep --color=auto oggsw
[root@gg23test ~]#We can infer from the above output that Service Manager process (PPID 609015) has started four child processes for adminsrvr, distsrvr, pmsrvr, recvsrvr. This is expected since both Service Manager and Oracle deployment are executing out of the same home. This can also be confirmed by observing the binary path ‘/dbgg/oggsw/oggorasw/bin’ which is common for all processes and configuration file (not to be confused with Configuration Service) path ‘/dbgg/oggservdep/oggoraservdep’ (OGG 23ai Oracle deployment home).
Next, I will start OGG 23ai PG deployment. This is where things get a bit interesting.
Service Manager confirms four services are started for OGG 23ai PG deployment.
[root@gg23test ~]# ps -ef|grep oggsw
oracle 6866 1 0 May20 ? 00:03:57 /dbgg/oggsw/oggorasw/bin/ConfigService -f /dbgg/oggservdep/oggoraservdep/var/lib/conf/ConfigService/ConfigService.json
oracle 609015 1 8 06:16 ? 00:04:19 /dbgg/oggsw/oggorasw/bin/ServiceManager --inventory '/dbgg/oggservdep/oggoraservdep/etc/conf'
oracle 613865 609015 5 06:56 ? 00:00:40 /dbgg/oggsw/oggorasw/bin/adminsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-adminsrvr-config.dat
oracle 613910 609015 0 06:56 ? 00:00:06 /dbgg/oggsw/oggorasw/bin/distsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-distsrvr-config.dat
oracle 613982 609015 0 06:56 ? 00:00:06 /dbgg/oggsw/oggorasw/bin/pmsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-pmsrvr-config.dat
oracle 614038 609015 0 06:56 ? 00:00:01 /dbgg/oggsw/oggorasw/bin/recvsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/ogg23oracle-recvsrvr-config.dat
oracle 615307 609015 0 07:07 ? 00:00:00 /dbgg/oggsw/oggpgsw/bin/adminsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/oggpgdep-adminsrvr-config.dat
oracle 615345 609015 0 07:07 ? 00:00:01 /dbgg/oggsw/oggpgsw/bin/distsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/oggpgdep-distsrvr-config.dat
oracle 615426 609015 0 07:07 ? 00:00:01 /dbgg/oggsw/oggpgsw/bin/pmsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/oggpgdep-pmsrvr-config.dat
oracle 615484 609015 0 07:07 ? 00:00:00 /dbgg/oggsw/oggpgsw/bin/recvsrvr --config /dbgg/oggservdep/oggoraservdep/var/run/oggpgdep-recvsrvr-config.dat
root 615843 608437 0 07:09 pts/1 00:00:00 grep --color=auto oggsw
[root@gg23test ~]#Observe the above output closely. We can observe those 4 processes (adminsrvr, distsrvr, pmsrvr, recvsrvr) are executing out of ‘/dbgg/oggsw/oggpgsw/bin/’ (OGG 23ai PG software home) path with configuration files (not to be confused with Configuration Service) out of ‘/dbgg/oggservdep/oggoraservdep’ location. This is my OGG 23ai Oracle deployment home!
There is only one ConfigService process with PID 6866 which was started along with the ServiceManager process. ConfigService configuration (for lack of better word) information is stored in the ‘/dbgg/oggservdep/oggoraservdep…./ConfigService.json’ file. This is the OGG 23ai Oracle deployment home.
We can conclude that only one ConfigService process exists for both deployments executing on same host and managed by same ServiceManager. What happens if deployment is on another host with another service manager? Can we use same backend table? More on this, some time in future.
Now it's time to verify the backend database table. Notice all information for OGG 23ai PG and OGG 23ai Oracle deployment is stored inside the same table.
SQL> /
name data added
----------------------------------------------------------- ------------------------------------------------------------------------------------- --------------------------------------------------
{"connectionString":"localhost:1523/freepdb1","tableName":"c##ggadmin.ggs_backen 18-MAY-24 07.49.12.476629 AM +00:00
ServiceManager {"$schema":"ogg:deployment","oggHome":"/dbgg/oggsw/oggorasw","id":"bfb44835-de85 18-MAY-24 07.49.12.604140 AM +00:00
configData {"$schema":"config:dataCollection","collectionSchema":"http://json-schema.org/dr 18-MAY-24 07.49.12.604614 AM +00:00
bfa325c4-oggadmin {"$schema":"ogg:credentials","userid":"oggadmin"} 18-MAY-24 07.49.25.863753 AM +00:00
ogg23oracle {"$schema":"ogg:deployment","id":"56fa711c-abce-40cf-9388-3fae7362d28f"} 18-MAY-24 07.49.33.631956 AM +00:00
ogg23oracle {"oggEtcHome":"/dbgg/oggdep/ogg23oracle/etc","oggVarHome":"/dbgg/oggdep/ogg23ora 18-MAY-24 07.49.33.705942 AM +00:00
adminsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 18-MAY-24 07.49.33.843764 AM +00:00
ogg23oracle:adminsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716360976,"usec" 18-MAY-24 07.49.33.859340 AM +00:00
bfa325c4-oggadmin {"$schema":"ogg:credentials","userid":"oggadmin"} 18-MAY-24 07.49.34.991245 AM +00:00
distsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 18-MAY-24 07.49.38.230425 AM +00:00
ogg23oracle:distsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716360977,"usec" 18-MAY-24 07.49.38.256225 AM +00:00
recvsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 18-MAY-24 07.49.38.525644 AM +00:00
ogg23oracle:recvsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716360979,"usec" 18-MAY-24 07.49.38.600401 AM +00:00
pmsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 18-MAY-24 07.49.38.991961 AM +00:00
ogg23oracle:pmsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716360978,"usec" 18-MAY-24 07.49.39.311560 AM +00:00
GLOBALS {"$schema":"ogg:config","lines":["GGSCHEMA c##ggadmin"]} 18-MAY-24 07.49.39.510977 AM +00:00
afd15321-test {"$schema":"ogg:credentials","userid":"c##ggadmin@localhost:1523/freepdb1"} 18-MAY-24 03.35.30.156880 PM +00:00
oggpgdep {"oggEtcHome":"/dbgg/oggdep/oggpgdep/etc","oggVarHome":"/dbgg/oggdep/oggpgdep/va 19-MAY-24 01.04.43.996981 PM +00:00
adminsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 19-MAY-24 01.04.44.334705 PM +00:00
oggpgdep:adminsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716361621,"usec" 19-MAY-24 01.04.44.617826 PM +00:00
bfa325c4-oggadmin {"$schema":"ogg:credentials","userid":"oggadmin"} 19-MAY-24 01.04.45.789753 PM +00:00
distsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 19-MAY-24 01.04.49.083189 PM +00:00
oggpgdep:distsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716361622,"usec" 19-MAY-24 01.04.49.385533 PM +00:00
recvsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 19-MAY-24 01.04.49.788613 PM +00:00
oggpgdep:recvsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716361624,"usec" 19-MAY-24 01.04.50.129834 PM +00:00
pmsrvr {"$schema":"ogg:service","config":{"hstsDetails":"max-age=31536000;includeSubDom 19-MAY-24 01.04.50.611911 PM +00:00
oggpgdep:pmsrvr {"$schema":"internal:taskHistory","history":[{"started":{"sec":1716361623,"usec" 19-MAY-24 01.04.51.173142 PM +00:00
GLOBALS {"$schema":"ogg:config","lines":["GGSCHEMA ggadminpg"]} 19-MAY-24 01.04.51.437970 PM +00:00
current {"$schema":"monitoring:datastore","type":"BDB"} 19-MAY-24 01.04.51.591559 PM +00:00
.......
.......
.......
86 rows selected.
SQL> l
1* select "name","data","added" from GGS_BACKENDTABLE order by 3Note — Some rows are omitted from above output above for brevity.
Individual OGG process entries can be identified by individual deployment names ‘ogg23oracle’ and ‘oggpgdep’. Observe that there is one entry for each deployment and its processes. This confirms both deployments are using same backend table and managed via single Service Manager process.
Cool Stuff. What do you think of this feature? Share your thoughts and comments in feedback. As always thank you for reading.
If you are using OCI GG , chances are you may never see Configuration Service in action, because it is orchestrated by Service Manager which is in turn managed by OCI. If you are using OGG on-prem this will be helpful to understand what it does, how it does and why it does.
Appendix
SQL query for OGG Oracle database backend table
col name form a65
col data form a95
col added form a50
set lines 400
set pages 1000
select systimestamp from dual;
select "name","data","added" from GGS_BACKENDTABLE order by 3;Response file for OGG 23ai PG deployment
Ready reckoner for technical jargon
