Wednesday, April 16, 2014

GoldenGate Splitting an existing table replicat group into multiple groups

Ok, the reason behind this would be no brainier as you obviously wanted to increase your replicat process to catch up heavy DML's.

As mentioned in earlier post, even after splitting the tables from one schema into 4 different groups, I still see that one table is a real killer (with about 40K records in 30 seconds. go figure) and its dying to catch up. So, how can I make this faster to catch up and reduce the lag time!!!

Well, its the very same approach as in my previous post but on thing that is different in this setup is:
In the new group prm file that you are adding, add the RANGE function like showed below:

Map schema_name.table_name , Target schema_name.table_name, FILTER (@RANGE (2, 3, Key_Column)) ; 

In the above, Range parameters 2 says its the 2nd group and 3 says that there 3 groups total for this table_name and Key_Column is an optional.

Note: This is best if you do it at Extract process instead of Replicat (read below).

Excerpt from Oracle Docs (if you were like me to search for the Doc :) ):
RANGE
Use the @RANGE function to divide the rows of any table across two or more Oracle
GoldenGate processes. It can be used to increase the throughput of large and heavily
accessed tables and also can be used to divide data into sets for distribution to different
destinations. Specify each range in a FILTER clause in a TABLE or MAP statement.
@RANGE is safe and scalable. It preserves data integrity by guaranteeing that the same row
will always be processed by the same process group.
@RANGE computes a hash value of the columns specified in the input. If no columns are
specified, the KEYCOLS clause of the TABLE or MAP statement is used to determine the columns
to hash, if a KEYCOLS clause exists. Otherwise, the primary key columns are used.
Oracle GoldenGate adjusts the total number of ranges to optimize the even distribution
across the number of ranges specified.
Because any columns can be specified for this function, rows in tables with relational
constraints to one another must be grouped together into the same process or trail to
preserve referential integrity.
NOTE Using Extract to calculate the ranges is more efficient than using Replicat.
Calculating ranges on the target side requires Replicat to read through the entire
trail to find the data that meets each range specification.
Syntax @RANGE (, [, ] [, ] [, ...])
Example 1 In the following example, the replication workload is split into three ranges (between three
Replicat processes) based on the ID column of the source acct table.
(Replicat group 1 parameter file)
MAP sales.acct, TARGET sales.acct, FILTER (@RANGE (1, 3, ID));
(Replicat group 2 parameter file)
MAP sales.acct, TARGET sales.acct, FILTER (@RANGE (2, 3, ID));
(Replicat group 3 parameter file)
MAP sales.acct, TARGET sales.acct, FILTER (@RANGE (3, 3, ID));
Argument Description
The range assigned to the specified process or trail. Valid values are 1, 2,
3, and so forth, with the maximum value being the value defined by ranges>.
The total number of ranges allocated. For example, to divide data into
three groups, use the value 3.
The name of a column on which to base the range allocation. This
argument is optional. If not used, Oracle GoldenGate allocates ranges
based on the table’s primary key.Column Conversion Functions
STRCAT
Oracle GoldenGate Windows and UNIX Reference Guide 462
..............................................................................
Example 2 In the following example, one Extract process splits the processing load into two trails.
Since no columns were defined on which to base the range calculation, Oracle GoldenGate
will use the primary key columns.
RMTTRAIL /ggs/dirdat/aa
TABLE fin.account, FILTER (@RANGE (1, 2));
RMTTRAIL /ggs/dirdat/bb
TABLE fin.account, FILTER (@RANGE (2, 2));
Example 3 In the following example, two tables have relative operations based on an order_ID column.
The order_master table has a key of order_ID, and the order_detail table has a key of order_ID and
item_number. Because the key order_ID establishes relativity, it is used in @RANGE filters for
both tables to preserve referential integrity. The load is split into two ranges.
(Parameter file #1)
MAP sales.order_master, TARGET sales.order_master,
FILTER (@RANGE (1, 2, order_ID));
MAP sales.order_detail, TARGET sales.order_detail,
FILTER (@RANGE (1, 2, order_ID));
(Parameter file #2)
MAP sales.order_master, TARGET sales.order_master,
FILTER (@RANGE (2, 2, order_ID));
MAP sales.order_detail, TARGET sales.order_detail,
FILTER (@RANGE (2, 2, order_ID));

No comments:

Post a Comment