Learning by Doing

In this post I am working with mongodb 3.4.6 on Redhat 7.x This was supposed to be a super straight forward exercise for me as I had done it a few times in the past and I'd never really encountered any issue until today. I started the mongod service as such, and I verified the process is running Note: every node in this exercise started the same way, same data mount, log mount, same port, etc. [root@msdlva-dsnopm02 apps]# mongod --port 30000 --dbpath /apps/data --logpath /apps/log/appDBlog.log & [root@msdlva-dsnopm02 apps]# ps -ef | grep mongod root 12854 23282 6 08:31 pts/0 00:00:00 mongod --port 30000 --dbpath /apps/data --logpath /apps/log/appDBlog.log root 12880 23282 0 08:32 pts/0 00:00:00 grep --color=auto mongod Then I logged into this node by issuing and started working on adding replica set member [root@msdlva-dsnopm02 apps]# mongo --port 30000 MongoDB shell version v3.4.6 connecting to: mongodb://127.0.0.1:30000/ MongoDB server version: 3.

This is the version of redhat I am running [root@msdlva-dsnops03 ~]# lsb_release -a LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: RedHatEnterpriseServer Description:    Red Hat Enterprise Linux Server release 7.3 (Maipo) Release:        7.3 Codename:       Maipo To start, we need to configure mongodb-enterprise repo https://docs.mongodb.com/manual/tutorial/install-mongodb-enterprise-on-red-hat/ This is the repo I used [root@msdlva-dsnops03 ~]# vi /etc/yum.repos.d/mongodb-enterprise.repo [mongodb-enterprise] name = MongoDB Enterprise Repository baseurl = https://repo.mongodb.com/yum/redhat/$releasever/mongodb-enterprise/3.4/$basearch/ gpgcheck = 1 enabled = 1 gpgkey = https://www.mongodb.org/static/pgp/server-3.4.asc [root@msdlva-dsnops03 ~]# yum repolist to verify the the repo can be c

This is just a quick demonstration of setting up extract, pump, and replicate. I prefer to edit the prm files at OS level, but we can also do GGSCI> edit param esource to enter the vi mode to create the prm file. Here are the sample prm files used in the demonstration: prm files: -------------------------------- Source: (2 processes) ~~~~~~~~~~~~~~~~ esource.prm ~~~~~~~~~ EXTRACT ESOURCE SETENV (ORACLE_SID=source) SETENV (ORACLE_HOME="/source/app/oracle/product/11.2.0.3") userid ggg, password ggg EXTTRAIL /ggSource/app/goldengate/product/12.1.0/dirdat/es --Resolves the TABLES to be replicated when they first encounter a dml operation WILDCARDRESOLVE DYNAMIC --Reduce the system I/O overhead of redolog reads EOFDELAY 5 TABLE mine.table; psource.prm ~~~~~~~~~ EXTRACT PDWDSPRD SETENV (ORACLE_SID=source) SETENV (ORACLE_HOME="/source/app/oracle/product/11.2.0.3") userid ggg, password ggg RMTHOST targetServer.example.net, MGRPO

This is the sample collection I am using in this example: db.getCollection('testCol').find({}).pretty() {     "_id" : ObjectId("59764121cd6857bf00a34809"),     "a" : 1,     "b" : 1 } {     "_id" : ObjectId("59764126cd6857bf00a3480a"),     "a" : 2,     "b" : 2 } {     "_id" : ObjectId("5976412ccd6857bf00a3480b"),     "a" : 3,     "b" : 3 } {     "_id" : ObjectId("59764294cd6857bf00a3480c"),     "a" : 4,     "b" : 4,     "arr" : [         {             "a" : 6,             "b" : 6         },         {             "a" : 7,             "b" : 7         }     ] } This will give me the number of element inside of the array: db.getCollection('testCol').aggregate([{$match:{$and:[{a:{"$eq": 4}}]}}, { $unwind : "$arr&

I found this post when I encountered some type issue and the better solution seems to be to convert the type in place. So I just did a quick experiment myself to make sure it works properly. to set up this experiment, first run db.testConvert.insert({typeField:NumberLong(1)}); db.testConvert.insert({typeField:NumberInt(1)}); db.testConvert.insert({typeField:1}); We verify now typeField has contain all three types, Int64, Int32, and Double Referencing from BSON type documentation, we know that Double is 1. Then we run the following var bulk = db.testConvert.initializeUnorderedBulkOp(), count=0; db.testConvert.find({"typeField":{"$type":1}}).forEach(function(doc) {     bulk.find({"_id":doc._id}).updateOne({         "$set":{"tmpField":NumberLong(doc.typeField.valueOf())},         "$unset":{"typeField":1}     });     bulk.find({"_id": doc._id}).updateOne({ "$rename": { &

This post is using MongoDB version 3.2.11 I ran into this issue after we performed db.col.insertMany and everything seemed to work fine without a problem, all records were there... until application connected to it and we started seeing a bunch error.  We noticed that the error is indicating some sort of type error, so we began to investigate,  Through a simple test we can see how this version of mongod behaves when we perform a simple insert of an integer db.test.insert({a:1}) and we look into what we had inserted db.test.find().pretty() {     "_id" : ObjectId("59724963cd6857bf00a347fb"),     "testInt" : 1 } the output seems to have indicated it's stored correctly as an integer, but when we look deeper We can see the type is stored as double instead of int Therefore, to insert either the int32 or int64 type, we should do db.test.insert({b:NumberInt(1)})   for int32, or db.test.insert({c:Numbe

I never encountered this issue until I needed to log into the mongod administered by others and I don't have the host level access. This can be done in both robomongo or via mongo shell Using  robomongo Location of mongod log can be found by using the getCmdLineOpts command: check the entry below --logpath for the location for mongod log.  via mongo shell mongo --host <hostname> --port <port number> MongoDB Etnerprise> db.adminCommand('getCmdLineOpts') /* 1 */ {     "argv" : [          "mongod",          "--port",          "21000",          "--replSet",          "avlrepdev",          "--dbpath",          "/data/mongodb/data",          "--logpath",          "/data/mongodb/log/mongodb.log",          "--logappend",          "--fork"     ],     "parsed" : {         "net" :

Goldengate version 12.1.2.1.0 At source: within your gg home directory ./ggsci GGSCI> dblogin userid <username>, password <password> GGSCI> add extract exsource, SOURCEISTABLE GGSCI> edit param exsource inside vi editor EXTRACT exsource SETENV (ORACLE_SID=orclSrc) SETENV (ORACLE_HOME="/u01/app/oracle/product/11.2.0.3") userid <gg username>, password <password> RMTHOST <remote hostname>, MGRPORT 7809 RMTFILE <Remote GG HOME>/dirdat/dp, MEGABYTES 1024, format level 4 PURGE --Reduce the system I/O overhead of redolog reads, <n> in second EOFDELAY 5 TABLE <schema>.<table name>; <ESC> then :wq to save and quit the editor GGSCI> start exsource At target: within your gg home directory ./ggsci GGSCI> add checkpointtable chk GGSCI> add replicat rtarget, exttrail <Remote GG HOME>/dirdat/dp, checkpointtable chk GGSCI> edit param rtarget inside vi edi

This post is to show that mongosqld schemaDirectory option requires the process to restart in order for the new drdl file to be picked up by the process. Identify mongosqld version [root@msdlva-dsnavl04 smartDrdl]# mongosqld --version mongosqld version v2.0.1 git version: 932107fa830951b6601b28cab2df234806cb3cd2 I have started the mongosqld as such  [root@msdlva-dsnavl04 smartDrdl]# ps -ef | grep mongosqld root     18506 14799  0 13:37 pts/1    00:00:01 mongosqld --addr=10.155.208.169:3307 --schemaDirectory=/apps/drdl --mongo-uri mongodb://xxx/?replicaSet=yyy --logPath /apps/biLog/mongoBI.log --sslPEMKeyFile=/apps/certs/mongodb.pem --sslAllowInvalidCertificates --auth --verbose=5 Let's check what files are in the drdl directory: [root@msdlva-dsnavl04 drdl]# ls -altr total 36 -rw-r--r--  1 root    root    11793 Jul 19 13:35 equipStatus.drdl -rw-r--r--  1 root    root    18764 Jul 19 13:36 personnel_status.drdl drwxr-xr-x 11 default de

This is a quick guide to uninstall and install any version of mongodb bi connector: First Identify the mongosqld is indeed present on this box. [root@msdlva-dsnavl04 ~]# mongosqld --version mongosqld version v2.2.0-rc0 git version: eface8f89d6b65fba09d831d6471b527ab508ab0 Identify where the mongosqld is installed [root@msdlva-dsnavl04 apps]# which mongosqld /usr/local/bin/mongosqld Mongosqld is usally installed with mongodrdl, therefore, we can just identify them by running [root@msdlva-dsnavl04 ~]# ls /usr/local/bin | grep mongo mongodrdl mongosqld or we can simply do  which mongodrdl, and we can find where the mongodrdl is installed Remove binaries [root@msdlva-dsnavl04 ~]# rm -r /usr/local/bin/mongo* rm: remove regular file ‘/usr/local/bin/mongodrdl’? y rm: remove regular file ‘/usr/local/bin/mongosqld’? y Download the versions you desire and copy onto the server.  I want to work with version 2.0,  tar -xvf m