MongoDB Atlas

  • Published on May 22, 2024
  • 7 minute(s) read
Prev Next

Overview

Syniti Replicate allows you to replicate data from relational database tables to MongoDB Atlas, a cloud-based document database, using the following methods:

  • Refresh or Snapshot Replication: A one-time complete replication from any major relational database source to MongoDB Atlas as a target, according to replication settings and scripts.

  • Mirroring or Change Data Capture Replication: A continuous update of a replicated table to MongoDB Atlas based on changes in the source database that have been recorded in the database server log. Any changes found in the log are applied to MongoDB Atlas as a target, according to replication settings and scripts.

MongoDB stores data in flexible consistently formatted documents. All records in MongoDB are associated with a specific collection inside a database. When replicating data using Syniti Replicate to MongoDB, the MongoDB file system is virtualized as relational tables (with columns and data types). You can create a target table and define a replication to a MongoDB target, applying mappings and expression logic as you would normally do with relational database targets. The MongoDB message is built internally using a JSON serialization of the entire record. Every replication must be associated with a MongoDB collection, so that Syniti Replicate can determine where to publish the message.

You can schedule the replication, identify the columns to be replicated, and add scripts to transform data during replication. Source databases include Oracle, Microsoft SQL Server, IBM Db2 for i, IBM Db2 LUW, Sybase, Informix, MySQL, and so on.

Use Syniti Replicate Management Center to:

  • Create source connections to RDBMS tables

  • Create MongoDB targets

  • Map RDBMS sources to MongoDB targets

  • Enable replication

Prerequisites

  • Ensure that you have downloaded the .NET Driver MongoDB Driver Version 2.19.0 for MongoDB v4.0. Click here to download.

  • Download and run the NuGet command line tool from NuGet Gallery.

  • Ensure that you have downloaded and installed Syniti Replicate.

    Note

    To download or register Syniti Replicate, log in to the support site, then click the required link in the Replicate section of the article. The Knowledge Platform Product Suites article acts as a hub to point to various resources. You can also submit a request for additional support.

Limitations

  • As MongoDB is not a standard relational database and only handles messages, it is not possible to open the content of a target table in the Management Center. To visualize data in your target system, use another application that can act as a consumer to receive the messages sent to MongoDB.

  • Syniti Verifier is not supported for MongoDB.

  • Data Encryption is not supported for MongoDB.

  • Data Masking is not supported in Syniti Replicate.

  • Databases, schemas, and tables names containing special characters like slash (/) or backslash (\) are not supported.

Setup Summary

This section lists a summary of all the steps required for setting up and using Syniti Replicate with MongoDB Atlas as a target.

Install .NET Provider for Source Database

  1. Install the .NET Provider using the downloaded MongoDB Driver Version 2.19.0.

  2. Create a local directory to deploy the files required, for example, C:\tmp\mongodb, and then save the nuget.exe file.

  3. From the DOS prompt with admin permissions, execute the below command. Make sure to execute the instruction from the right directory.

    nuget install MongoDB.Driver -Version 2.19.0 -Framework net480 -OutputDirectory C:\tmp\Mongodb

    This command downloads all assemblies into the specified folder. However, there are separate folders for each of the assemblies.

  4. Open each folder and extract the version of the .dll files accordingly:

    • Valid for .NET framework 4.6 or earlier (netxx), if available, or

    • Use the latest .NET Standard framework (netstandardxx) version

      Note

      For .NET Framework 4.6 installations, you cannot use the net471 folder because it is higher than 4.6, but you can use net45. If that were not included, you would select netstandard2.0.

  5. Copy the following extracted assemblies to <SynitiDR-install-dir>/Plugins/MongoDB/:

    • DnsClient.1.6.1\lib\netstandard2.0\DnsClient.dll

    • Microsoft.Bcl.AsyncInterfaces.5.0.0\lib\netstandard2.0\Microsoft.Bcl.AsyncInterfaces.dll

    • Microsoft.Extensions.Logging.Abstractions.2.0.0\lib\netstandard2.0\Microsoft.Extensions.Logging.Abstractions.dll

    • Microsoft.Win32.Registry.5.0.0\lib\netstandard2.0\Microsoft.Win32.Registry.dll

    • MongoDB.Bson.2.19.0\lib\netstandard2.0\MongoDB.Bson.dll

    • MongoDB.Driver.Core.2.19.0\lib\netstandard2.0\MongoDB.Driver.Core.dll

    • MongoDB.Driver.2.19.0\lib\netstandard2.0\MongoDB.Driver.dll

    • MongoDB.Libmongocrypt.1.7.0\lib\netstandard2.0\MongoDB.Libmongocrypt.dll

    • SharpCompress.0.30.1\lib\netstandard2.0\SharpCompress.dll

    • Snappier.1.0.0\lib\netstandard2.0\Snappier.dll

    • System.Buffers.4.5.1\lib\netstandard2.0\System.Buffers.dll

    • System.Memory.4.5.4\lib\netstandard2.0\System.Memory.dll

    • System.Numerics.Vectors.4.5.0\lib\netstandard2.0\System.Numerics.Vectors.dll

    • System.Runtime.CompilerServices.Unsafe.5.0.0\lib\netstandard2.0\System.Runtime.CompilerServices.Unsafe.dll

    • System.Security.AccessControl.5.0.0\lib\netstandard2.0\System.Security.AccessControl.dll

    • System.Security.Principal.Windows.5.0.0\lib\netstandard2.0\System.Security.Principal.Windows.dll

    • System.Text.Encoding.CodePages.5.0.0\lib\netstandard2.0\System.Text.Encoding.CodePages.dll

    • System.Threading.Tasks.Extensions.4.5.4\lib\netstandard2.0\System.Threading.Tasks.Extensions.dll

    • ZstdSharp.Port.0.6.2\lib\netstandard2.0\ZstdSharp.dll

Syniti Replicate Setup

Complete the following steps in the Syniti Replicate Management Center:

  1. In the Metadata Explorer, create a source connection to your RDBMS.

  2. Create a target connection using the MongoDB Atlas option in the Database field.

  3. Create a replication.

Start Replicating

Start the Replication Agent in the Syniti Replicate Service Monitor.

Steps for Replicating Tables

Syniti Replicate support for replicating relational data to MongoDB Atlas allows you to set up replications using either Refresh or Mirroring modes. The steps below explain how to replicate data from a relational database to MongoDB. Refer to All Source and Target Options for the latest list of supported databases.

Set up a Target Connection to MongoDB

  1. Select the Targets node in the Metadata Explorer.

  2. Right-click and select Add New Connection.

  3. In the Add Target Connection Wizard, select MongoDB Atlas from the Database list. The Provider and Assembly fields are automatically filled out for you.

  4. Click Next.

  5. On the Set Connection String page, set the required and optional properties as described in the below tables. The table displays only properties specific for use with Syniti Replicate. You can also set additional MongoDB properties as needed.

  6. Click Next to move on to the Select tables page.
    At this point, there is no text output structure available to display. You can add the information after completing the Add Target Connection wizard. Refer to Add Target Connection Wizard for more information.

  7. Click Next to display the Summary page.

  8. Click Finish to complete the wizard.

Connection Properties - Required Information

Property

Description

Server

Server name or IP address of the MongoDB instance.

Port

The port number of the MongoDB server.

Use DNS Seedlist

If set to false, the standard connection format is used. If set to true, a DNS-constructed seed list is used. Using DNS to construct the available servers list allows more flexibility of deployment and the ability to change the servers in rotation without reconfiguring clients.

User ID

Username for MongoDB

Password

Password for MongoDB

Database

MongoDB database name

Output Folder

The schema name and location to hold config files for the MongoDB objects.

Connection Properties - Optional Information

Property

Description

Default Collection

Default collection to use in replication. If a valid collection is set, all MongoDB documents will be added to this collection, otherwise a new collection will be created for each replicated table.

Create Collection per Table

If set to true, each replicated table will create a collection using the table name.

In this case, the default collection property will be ignored. The collection can be changed using the Collection Name property of the table.

Replica Set

Specifies the name of the replica set, if the mongod is a member of a replica set. When connecting to a replica set it is important to give a seed list of at least two mongod instances. If you only provide the connection point of a single mongod instance, and omit the ReplicaSet, the client will create a standalone connection.

Use SSL

If set to true, the provider will initiate the connection with TLS/SSL. The default value is false.

Connect Timeout

The time in milliseconds to attempt a connection before timing out. The default value is 0 (never to timeout).

Socket Timeout

The time in milliseconds to attempt a send or receive on a socket before the attempt times out. The default value is 0 (never to timeout).

Max Pool Size

The maximum number of connections in the connection pool. The default value is 100.

Min Pool Size

The minimum number of connections in the connection pool. The default value is 0.

Max Idle Time

The maximum number of milliseconds that a connection can remain idle in the pool before being removed and closed.

Read Concern Level

The level of isolation. Can accept one of the following values:

  • Local: (Default value) It read concern can read data which may be subsequently rolled back during replica set failovers

  • Majority: Enables multiple threads to perform reads and writes on a single document

  • Linearizable: It read concern can read data which may be subsequently rolled back during replica set failovers

  • Available: It read concern can read data which may be subsequently rolled back during replica set failovers

Read Preferences

Specifies the replica set read preference for this connection. The read preference values are the following:

  • Primary

  • PrimaryPreferred

  • Secondary

  • SecondaryPreferred

  • Nearest.

The default value is Primary, which sends all read operations to the replica set. Multi-document transactions that contain read operations must use read preference primary. All operations in a given transaction must route to the same member.

Auth Source

Specify the database name associated with the user's credentials. AuthSource defaults to the database specified in the connection string. For authentication mechanism that delegates credential storage to other services, the authSource value should be $external as with the PLAIN(LDAP) and GSSAPI(Kerberos) authentication mechanism. MongoDB will ignore authSource values if the connection string specifies no username.

Auth Mechanism

Specify the authentication mechanism that MongoDB will use to authenticate the connection. Possible values:

  • SCRAM - SHA-1

  • SCRAM - SHA-256 (Added in MongoDB 4.0)

  • MONGODB-CR (removed in MongoDB 4.0)

  • MONGDB X509

  • GSSAPI (Kerberos)

  • PLAIN (LDAP SASL)

Retry Writes

Enables you to retry writes. Possible values are true/false. The default value for 3.6 drivers is false.

Use One Writer Per Group

Indicates if each replication group generates a common MongoDB writer or one writer for each replication in the group

Extended Properties

Additional properties not listed in the grid. Enter the properties as a list of "property=value" string separated by (;).

Editing Collection Name

If you select the table and edit the Properties, there is a new property called Collection Name. Collection Name can be edited. If blank, Syniti Replicate uses the Default Collection property of the connection to determine the collection to deliver the records. Otherwise, it uses the collection indicated in this field.

Note that if you replicate a table that has a blank collection value both here and in the connection, the Replication Agent generates an error and disables the replication, unless you set the Create Collection Per Table property to True, in which a collection will be create automatically.