Unix quick tip - tar with progress

-
tar with progress bar

tar cf - /folder-with-big-files -P | pv -s $(du -sb /folder-with-big-files | awk '{print $1}') | gzip > big-files.tar.gz

to install pv

first we need to add the repo

create

/etc/yum.repos.d/ivarch.repo:
[ivarch]
name=RPMs from ivarch.com
baseurl=http://www.ivarch.com/programs/rpms/$basearch/
enabled=1
gpgcheck=1

import public key
rpm --import http://www.ivarch.com/personal/public-key.txt

yum install pv -y


trying it out


Comments

Post a Comment

Popular posts from this blog

MongoDB tip: 4 ways to modify replica set configuration

-
This is just a quick guide on how to modify your replica set settings in 4 different ways. This guide is assuming you have a functional replica set ready, it doesn't matter if it's a single node or multi-node replica set. In each of the four method, I am looking to modify the priority of the _id=0 node Method 1: Via Mongo Shell Login to your Mongo shell,  simply type in mongo , assuming default port is being used and authentication is not enabled [in Mongo shell] cfg = rs.conf() cfg.members[0].priority=99 rs.reconfig(cfg) # "ok":1 indicates the new configuration has been set  MongoDB Enterprise minvera:PRIMARY> rs.reconfig(cfg) {         "ok" : 1,         "operationTime" : Timestamp(1528464949, 1),         "$clusterTime" : {                 "clusterTime" : Timestamp(1528464949, 1),                 "signature" : {                         "hash" : BinData(0
Read more

MongoDB Quick Note: BI Connector Issue

-
This is just a quick note of what can stop mongosqld from working properly that I wasn't aware of. Background: My work runs everything mongodb inside of docker containers, so sometimes it's not as straight forward to see immediately of what's preventing your container from working properly What happened: I ran into issue that my container does not stay up after I updated my drdl to reflect new fields introduced to the underlying collection. How to debug: Go to the host server, copy out entrypoint.sh and add sleep 100000000 to before mongosqld is executed. This force the container to stay up for us to look into issue. Cause of the issue: When I updated the drdl, I used tab instead of space, which is causing the mongsqld to error out Fix: Just replace tab with space and everything is back to normal
Read more

MongoDB Tips: Kill long running processes

-
This is just a quick and simple script to find all active opid that has been running over 10 seconds and they are running against the databases and collections you can specify in the "ns" filter. It's best to first run db.currentOp(    {      "active" : true,      "secs_running" : { "$gt" : 10 },      "ns" : {$in:["equipment.detachment","personnel.detail","personnel.unavailability","personnel.detachment"]}    } ).inprog.forEach(function (doc) { print(doc.opid); }) and verify your output before you execute the kill command in my experience to make sure you are killing the processes that are meant to be killed. Once verifed, simply run: db.currentOp(    {      "active" : true,      "secs_running" : { "$gt" : 10 },      "ns" : {$in:["<db1>.<col1>","<db2>.<col2>","<dbn>.<coln>
Read more