How to understand Exim logs files.
In cpanel exim mail logs are stored in following files
/var/log/exim_mainlog -: All the logs are recorded into the exim_mainlog file
/var/log/exim_rejectlog -: Rejected emails logs are recorded into the exim_rejectlog
/var/log/exim_paniclog -: If the exim server stuck or fails then those logs will be recorded into the exim_paniclog file.
Log line flags:
—————
<= Shows arrival of a message for incoming mail.
=> Shows a normal message delivery for outgoing mail
-> Shows additional address for the same delivery, i.e. an Email forwarder.
*> Shows delivery suppressed by -N**delivery failed; address bounced.
== Shows delivery deferred; temporary problem.
** Shows delivery failed; address bounced.
From the email queue, we can find the log details of an email by using the command “exim -Mvl Message-ID”
A summary of the field identifiers that are used in log lines:
R= The address immediately following “<=” is the sender address. A bounce message is shown with the sender address “<>”, and if it is locally generated, this is followed by an item of the form R=<message id>.
T= The relay used to transmit the message. Example: T=remote_smtp T=local_delivery
H= Represents the host name and IP address.
U= The MTA used.
I= local interface used.
P= This is the return path on delivery.
A= If A= is present, then SMTP AUTH was used for the delivery.
S= Is the delivery size of the message.
ID= Represents the incoming message ID
T= Topic / Subject
from :From whom the mail was received
for : Who the email is for.
C : SMTP confirmation on delivery.
D : Duration of “no mail in SMTP session”.
S : size of message.
X :TLS cipher suite
DT:on => lines: time taken for a delivery
Email of users are stored in “/home/user/mail/<domain>/<emailuser>”
/home/username/mail(mail folder)
/home/username/etc (configuration files)
/domain.com
maildirsize (quota/usage summary, expendable)
cur (default inbox for read email)
new (default email for unread email)
Understanding localdomain and remotedomain
In order to send the mails the website and email for a domain need not be in same server. DNS is used to point the services to the correct place. Exim, cPanel SMTP service needs a little help beyond DNS in order to know how to handle mail that is generated locally. This is where /etc/localdomains and /etc/remotedomains are usefull.
/etc/localdomains :
If the MX record of the domain is pointed to same server then there must be entry for that domain under /etc/localdomains file for local delivery of email.
/etc/remotedomains :
If you are using remote mail server then entry for that domain must be removed from /etc/localdomains files and should added in /etc/remotedomains files for proper routing of emails.
Setting this option during account creation
——————————————-
Local Mail Exchanger:-cPanel will place the domain in localdomains.
Remote Mail Exchanger :- It will place the domain in remotedomains.
Backup Mail Exchanger:- The domain will be added to localdomains but will only accept mail if there are no other mail servers available.
Automatically Detect Configuration:- It will check the DNS Mail Exchanger records to see where the DNS is pointed, and make the configuration based off of that.
Changing the exchanger after account creation in WHM
—————————————————–
In WHM, navigate to “Edit MX Entry” under “DNS Functions” in the left menu list.
From there, simply select the domain name you wish to edit, and then select the desired mail exchanger (Local Mail Exchanger, Backup Mail Exchanger, or Remote Mail Exchanger) to change the mail routing settings for the domain:
Changing the exchanger after account creation via SSH
———————————————————-
Modifying a domain to use a different exchanger is as simple as SSHing into the server and editing the files; simply remove the domain from one file and add it to the other. It is that simple. No need to restart services afterward.
How to change sites MX record to point mail to another domain or server.
————————————————————————-
An MX Record is used by a mail server to route incoming email. Changing your MX record is usually done to redirect email to a remote server.
To point the MX to another domain do the following.
1. Log into your WHM.
2. Click the Edit DNS Zone.
3. Select the domain name from the drop-down list.
4. Change the MX from domain.com to mail.domain.com.
5. Click the save button.
If you want to point it to another server using the same domain:
1. Log into your WHM.
2. Click the Edit DNS Zone.
3. Pick the domain you wish to edit.
4. Change the MX from domain.com to mail.domain.com.
5. mail.domain.com should have the A entry of the external mail server IP address.
6. Click the save button.
Note: You need to change the domain from /etc/localdomains to /etc/remotedomains for sending mails when your mail service is hosted on another server.
Authentication in mongodb
Mysql databases allow username and password. Similarly if you require to keep authentication for your mongodb database as well, please do the following. Suppose, I have a database named abc, and I need to create a user named abc with password abc123.
First, create an admin user.
root@abc:~# mongo
MongoDB shell version: 2.4.10
connecting to: test
> use admin
> db.addUser(“admin”, “admin”)
> use abc (Switch to the database abc)
> db.addUser(“abc”, “abc123″)
Quit the mongoshell and stop mongo
root@abc:~# /etc/init.d/mongodb stop
root@abc:~# vi /etc/mongodb.conf and make the following change
auth = true
root@abc:~#/etc/init.d/mongodb start
Now we need to test if authentication works. Which means, if not authenticated, you should not be able to saw the details of the database.
root@abc:~# mongo
MongoDB shell version: 2.4.10
connecting to: test
> use abc
> show collections
“$err” : “not authorized for query on abc.system.namespaces”,
“code” : 16550
Now it shows the error that you are not authorized to view the collections. Now, authenticate with the details you created earlier and try.
> db.auth(“abc”, “abc123″)
>show collections
testcollection
system.indexes
This is how you create a user for mongodb database.
Accessing Gmail with Roundcube
Requirement : I want to access my gmail under my own domain with the help of webmail.
Say my domain name is abc.com , I want to access it as http://abc.com/mail
Suppose I am on a cpanel server. Download and install Roundcube as follows.
root@me [/home/abc/public_html]# wget http://downloads.sourceforge.net/project/roundcubemail/roundcubemail/1.0.0/roundcubemail-1.0.0.tar.gz
root@me [/home/abc/public_html]# tar -xzvf roundcubemail-1.0.0.tar.gz
root@me [/home/abc/public_html]# mv roundcubemail-1.0.0 mail
root@me [/home/abc/public_html]# chown -R abc:abc mail
root@me [/home/abc/public_html]# cd mail
Now login to your cpanel and create a database for roundcube installation, create a database user and assign privileges. This can all be done from cpanel.
Once done, go to the installer link as http://abc.com/mail/installer, follow the on screen instructions, make sure to give the database details correct and do not overwrite or create any conflict with the existing roundcube installation that comes with cpanel.
Once the installation is done, there are two files which requires attention.
root@me [/home/abc/public_html/mail]# cd /home/abc/public_html/mail/config
Make sure the config.inc.php and defaults.inc.php are populated with the correct database details.
root@me [/home/abc/public_html/mail/config]# vi defaults.inc.php and edit the following fields
// IMAP Settings
$rcmail_config[‘default_host’] = ‘ssl://imap.gmail.com';
$rcmail_config[‘defaul_port’] = 993;
$rcmail_config[‘imap_auth_type’] = null;
$rcmail_config[‘username_domain’] = ‘yourdomainname'; // Give the domain name that use googleapps. If you are using gmail account, give gmail.com
$rcmail_config[‘mail_domain’] = ‘yourdomainname'; // Give the domain name that use googleapps. If you are using gmail account, give gmail.com
// SMTP Settings
$rcmail_config[‘smtp_server’] = ‘ssl://smtp.gmail.com';
$rcmail_config[‘smtp_port’] = 465;
$rcmail_config[‘smtp_user’] = ‘%u';
$rcmail_config[‘smtp_pass’] = ‘%p';
//MBOX Settings
$rcmail_config[‘drafts_mbox’] = ‘[Gmail]/Drafts';
$rcmail_config[‘junk_mbox’] = ‘[Gmail]/Spam';
$rcmail_config[‘sent_mbox’] = ”;
$rcmail_config[‘trash_mbox’] = ”;
Save and quit the file. Remove the installer folder.
root@me [/home/abc/public_html/mail/config]# rm -rf /home/abc/public_html/mail/installer
Now access your gmail through your roundcube installation.
http://abc.com/mail
Username : test if my gmail account is test@gmail.com
Password : test123 if my gmail pass is test123
Migrating emails from mailenable to exim
If you want to copy emails from a windows mailenable server to linux server with exim, you can do it as follows.
I am copying the email account test@abc.com to new server. Its password is test123c on both servers.
root@abc [~]# cd /usr/local/src/
root@abc [/usr/local/src]# wget http://home.arcor.de/armin.diehl/imapcopy/imapcopy.tar.gz
root@abc [/usr/local/src]# tar -xvf imapcopy.tar.gz
root@abc [/usr/local/src]# cd imapcopy
root@abc [/usr/local/src/imapcopy]# ls
./ ../ imapcopy* ImapCopy.cfg
root@abc [/usr/local/src/imapcopy]# cp -pv ImapCopy.cfg ImapCopy.cfg.original
`ImapCopy.cfg’ -> `ImapCopy.cfg.original’
root@abc [/usr/local/src/imapcopy]# vi ImapCopy.cfg
You need to edit the following values to make the migration smoother.
##############
# Sourceserver
##############
SourceServer 1.2.3.4 (Source Server IP Address)
SourcePort 143
###################
# Destinationserver
###################
DestServer localhost
DestPort 143
#############################
# List of users and passwords
#############################
# SourceUser SourcePassword DestinationUser DestinationPassword
Copy “test@abc.com” “test123c” “test@abc.com” “test123c”
[/stextbox]
Make sure you are able to connect to port 143 of source server from destination server
[stextbox id=”info”]
root@abc [/usr/local/src/imapcopy]# telnet 1.2.3.4 143
Trying 1.2.3.4…
Connected to abc.greproot.com (1.2.3.4).
Escape character is ‘^]’.
* OK IMAP4rev1 server ready at 05/07/14 05:46:17
^]
telnet> q
Connection closed.
[/stextbox]
Now test the configuration
[stextbox id=”info”]
root@abc [/usr/local/src/imapcopy]# ./imapcopy -t
IMAPCopy 1.04 – 2009/07/18 [compiled with FreePascal]
written 2001-2009 by Armin Diehl <ad@ardiehl.de>
Running on Linux
Login on sourceserver as test@abc.com OK
Login on destinationserver as test@abc.com OK
Getting folderlist on sourceserver OK, found 1 folder
1 User processed, 0 Messages copied, 0 Error(s)
0 Folder(s) created, 0 Folder create errors, 0 Folder not copied
[/stextbox]
The configuration looks fine without any errors. Now we can start copying the email account contents.
[stextbox id=”info”]
root@abc [/usr/local/src/imapcopy]#./imapcopy
IMAPCopy 1.04 – 2009/07/18 [compiled with FreePascal]
written 2001-2009 by Armin Diehl <ad@ardiehl.de>
Running on Linux
Login on sourceserver as test@abc.com OK
Login on destinationserver as test@abc.com OK
Getting folderlist on sourceserver OK, found 1 folder
Getting List of messages in “INBOX” OK, 7 Messages found
Processing Folder INBOX
7 Messages copied, 0 Errors
1 User processed, 7 Messages copied, 0 Error(s)
0 Folder(s) created, 0 Folder create errors, 0 Folder not copied
[/stextbox]
Now the mails have been copied. Try accessing webmail of the account test@abc.com and see if mails are displayed fine.
Xen Virtualization in Centos 6
Xen virtualization steps are given below.
1. Make sure that SELinux is disabled or permissive.
vi /etc/sysconfig/selinux
and set “SELINUX=disabled”. Then reboot server.
2. Creating A Network Bridge.
We need to set up a network bridge on our server so that our virtual machines can be accessed from other hosts as if they were physical systems in the network.
To do this, we install the package bridge-utils
=========
yum install bridge-utils
=========
… and configure a bridge. Create the file /etc/sysconfig/network-scripts/ifcfg-br0 (please use the IPADDR, PREFIX, GATEWAY, DNS1 and DNS2 values from the/etc/sysconfig/network-scripts/ifcfg-eth0 file); make sure you use TYPE=Bridge, not TYPE=Ethernet:
vi /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE="br0" NM_CONTROLLED="yes" ONBOOT=yes TYPE=Bridge BOOTPROTO=none IPADDR=192.168.0.100 PREFIX=24 GATEWAY=192.168.0.1 DNS1=8.8.8.8 DNS2=8.8.4.4 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System br0" Modify /etc/sysconfig/network-scripts/ifcfg-eth0 as follows (comment out BOOTPROTO, IPADDR, PREFIX, GATEWAY, DNS1, and DNS2 and add BRIDGE=br0): vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0" NM_CONTROLLED="yes" ONBOOT=yes HWADDR=00:1E:90:F3:F0:02 TYPE=Ethernet #BOOTPROTO=none #IPADDR=192.168.0.100 #PREFIX=24 #GATEWAY=192.168.0.1 #DNS1=8.8.8.8 #DNS2=8.8.4.4 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth0" UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 BRIDGE=br0 Restart the network: /etc/init.d/network restart and run "ifconfig", it should network bridge. [root@server1 ~]# ifconfig br0 Link encap:Ethernet HWaddr 00:1E:90:F3:F0:02 inet addr:192.168.0.100 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::21e:90ff:fef3:f002/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:17 errors:0 dropped:0 overruns:0 frame:0 TX packets:29 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1196 (1.1 KiB) TX bytes:2794 (2.7 KiB) eth0 Link encap:Ethernet HWaddr 00:1E:90:F3:F0:02 inet6 addr: fe80::21e:90ff:fef3:f002/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4554 errors:0 dropped:0 overruns:0 frame:0 TX packets:3020 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:6249612 (5.9 MiB) TX bytes:254928 (248.9 KiB) Interrupt:25 Base address:0x6000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:3 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1304 (1.2 KiB) TX bytes:1304 (1.2 KiB) 3. Installing XEN: First check if your CPU supports hardware virtualization - if this is the case, the command egrep '(vmx|svm)' --color=always /proc/cpuinfo
should display something, e.g. like this:
[root@server1 ~]# egrep ‘(vmx|svm)’ –color=always /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall
nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy misalignsse
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall
nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy misalignsse
[root@server1 ~]#
If nothing is displayed, then your processor doesn’t support hardware virtualization. This means you can use only paravirtualization with Xen, but not hardware virtualization.
As CentOS 6 is based on RedHat 6, and RedHat has dropped support for Xen in version 6, we need to get Xen from a third-party repository. We can enable the repo as follows:
yum install wget
yum install http://au1.mirror.crc.id.au/repo/kernel-xen-release-6-3.noarch.rpm
yum install kernel-xen xen
This installs Xen and a Xen kernel on our CentOS system.
Before we can boot the system with the Xen kernel, please check your GRUB bootloader configuration. We open /boot/grub/menu.lst:
vi /boot/grub/menu.lst
The first listed kernel should be the Xen kernel that you’ve just installed:
[...]
title CentOS (2.6.32.54-1.el6xen.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM
initrd /initramfs-2.6.32.54-1.el6xen.x86_64.img
[...]
We need to modify that section so that the Xen hypervisor gets loaded first. In the kernel /vmlinuz... line, replace the first word kernel with module. Do the same in the next line - replace the first word initrd with module in the initrd /initramfs... line. Then add the line kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin after the root line and before the first module line (if you have more than one CPU core, you can specify another number than 1 fordom0_max_vcpus). The final kernel section should look like this:
[...]
title CentOS (2.6.32.54-1.el6xen.x86_64)
root (hd0,0)
kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin
module /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM
module /initramfs-2.6.32.54-1.el6xen.x86_64.img
[...]
Change the value of default to 0 (so that the first kernel (the Xen kernel) will be booted by default):
[...] default=0 [...] The complete /boot/grub/menu.lst should look something like this:
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/mapper/VolGroup00-LogVol00
# initrd /initrd-[generic-]version.img
#boot=/dev/sde
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32.54-1.el6xen.x86_64)
root (hd0,0)
kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin
module /vmlinuz-2.6.32.54-1.el6xen.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM
module /initramfs-2.6.32.54-1.el6xen.x86_64.img
title CentOS (2.6.32-220.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/mapper/VolGroup00-LogVol00 rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=de rd_LVM_LV=VolGroup00/LogVol01 rd_LVM_LV=VolGroup00/LogVol00 rd_NO_DM
initrd /initramfs-2.6.32-220.el6.x86_64.img
Before we reboot, we install the libvirt and python-virtinst (which contains the virt-install tool which we will use later on to install Xen VMs) packages:
yum install libvirt python-virtinst
Because the libvirt package from CentOS 6/RedHat 6 has no support for Xen, we must rebuild it with Xen support. To do this, we install a few prerequisites now:
yum groupinstall 'Development Tools'
yum install python-devel xen-devel libxml2-devel xhtml1-dtds readline-devel ncurses-devel libtasn1-devel gnutls-devel augeas libudev-devel libpciaccess-devel yajl-devel sanlock-devel libpcap-devel libnl-devel avahi-devel libselinux-devel cyrus-sasl-devel parted-devel device-mapper-devel numactl-devel libcap-ng-devel netcf-devel libcurl-devel audit-libs-devel systemtap-sdt-devel
Now reboot machine to load new kernel
The system should now automatically boot the new Xen kernel. After the system has booted, we can check that by running
uname -r
[root@server1 ~]# uname -r
2.6.32.54-1.el6xen.x86_64
[root@server1 ~]#
So it’s really using the new Xen kernel!
a. xm list
b. to check if Xen has started. It should list Domain-0 (dom0):
[root@server1 ~]# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 1024 1 r—– 18.9
[root@server1 ~]#
Instead of using the xm command, I will from now on use the virsh command to manage Xen VMs. This is the preferred way as we are using libvirt.
virsh list
should show this:
[root@server1 ~]# virsh list
Id Name State
———————————-
0 Domain-0 running
[root@server1 ~]#
4. Paravirtualization:
CentOS has a nice tool called virt-install with which we can create virtual machines for Xen. To start it, we simply run
virt-install –prompt –network bridge=br0 –virt-type=xen
The tools asks a few questions before it creates a virtual machine. I want to call my first virtual machine vm01, with 1024MB RAM and a disk size of 10GB. I want to store it in the file /var/lib/libvirt/images/vm01.img:
Would you like a fully virtualized guest (yes or no)? This will allow you to run unmodified operating systems.
<– no
What is the name of your virtual machine?
<– vm01
How much RAM should be allocated (in megabytes)?
<– 1024
What would you like to use as the disk (file path)?
<– /var/lib/libvirt/images/vm01.img
How large would you like the disk (/vm/vm01.img) to be (in gigabytes)?
<– 10
What is the install URL?
<– http://ftp.tu-chemnitz.de/pub/linux/centos/6.2/os/x86_64
Regards,
Jeen
Sharding in mongodb
Just putting together, what we have done in sharding so that we dont forget it later. I shall update this doc as and when we have more details.
Sharding means, distributing data across multiple servers.Basically, mongodb sharding requires three things
1) a config server which stores the metadata which knows where the data resides
2) a query router server is the server to which the application actually communicates. It contacts the config servers to find in which shard the data resides and retrieves the data to the application.
3) shard servers – this consists of a subset of the entire data, distributed across multiple servers
In our case, for test purpose, we used the minimal number of servers. Mongod eats up ram hence, it will be good to use servers with somewhat good specifications. We used
1) 1 config servers
2) 1 query router server. We combined config server and query router server into one, hence these two required only 1 server.
3) In order to see how sharding actually works, we needed 2 shard servers. So a total of 3 servers
Install mongo in all servers as mentioned in http://greproot.com/install-mongodb-centos/
Setting up config server
————————–
Hostname of server chosen as config.mongotest.com
mkdir /mongo-metadata – create a folder for the mongo metadata
Now start mongo config server as follows. Make sure to use the port as 27019. Whatever number of config servers you use, you need to make sure the path and port are same for all.
mongod –configsvr –fork –logpath=/var/log/mongo/mongod.log –dbpath /mongo-metadata –port 27019
Setting up query router
—————————-
Please note I chose config and query router servers to be same. If you have an alternate server, use it as queryrouter server. Query router use the mongos service. Mongos runs on port 27017.
mkdir /queryrouter_log
Start mongos as follows.
mongos –fork –logpath /queryrouter_log/query.log –configdb config.mongotest.com:27019
Shard Servers
—————–
Hostnames chosen are shard1.mongotest.com and shard2.mongotest.com. Just start mongodb in both servers and it will run on port 27017
We dont have to setup shard servers separately. Just login to any one shard server and you can setup all shards from there itself.
Login to any shard server as root. Connect to the query router server from there as follows.
mongo –host config.mongotest.com –port 27017
above command connects to the mongo shell of queryrouter server, at port 27017 which runs mongos
mongo –host config.mongotest.com –port 27017
MongoDB shell version: 2.4.9
connecting to: config.mongotest.com:27017/test
Add the two shard servers first
mongos> sh.addShard( “shard1.mongotest.com:27017″ )
mongos> sh.addShard( “shard2.mongotest.com:27017″ )
Create a new database
mongos> use divya_test
switched to db divya_test
Enable Sharding for that db
mongos> sh.enableSharding(“divya_test”)
{ “ok” : 1 }
Create a new collection test with an index _id
mongos> db.test.ensureIndex( { _id : “hashed” } )
Now shard this collection using a hashed shard key(i am not very sure of how shard keys has to be selected)
sh.shardCollection(“divya_test.test”, { “_id”: “hashed” } )
You can see the status of the shards by issuing the following command
mongos>sh.status()
Now, try adding some data to the collection and check both shard servers. You will see the data is spread across those servers.
mongos> db.test.save({_id:1})
mongos> db.test.save({_id:2})
mongos> db.test.save({_id:3})
mongos> db.test.find()
{ “_id” : 1 }
{ “_id” : 2 }
{ “_id” : 3 }
Multiple php versions on Centos
Hi , you may come across situations to have multiple php versions installed on a server. Here I will explain a simple method to accomplish this. Lets take php versions 5.2 and 5.4 and I am going to install it in different locations than default installation directory.
Note for 2 versions of php one version should be compiled as fcgi.
1. Install php 5.2:
a. Download php tar file from “http://php.net/downloads.php”
Here I am going to compile this php as fastcgi and using custom installation directory /opt/php52. Configure command is given below.
==========
# ./configure –prefix=/opt/php52 –with-config-file-path=/opt/php52 –with-curl –enable-cli –enable-fastcgi –enable-discard-path –enable-force-cgi-redirect –with-mysql –with-mysqli –enable-bcmath –enable-ftp –enable-magic-quotes –with-pear –enable-sockets –with-zlib –with-gd –with-jpeg-dir=/usr/lib –with-libdir=lib64 –with-bz2 –enable-calendar –with-curl –enable-dbase –enable-exif –with-gettext –with-gmp –enable-mbstring –with-mcrypt –with-mhash –with-ncurses=shared –with-openssl –enable-pcntl –with-pdo-mysql –with-pspell –with-readline –enable-shmop –with-snmp=shared –enable-sysvsem –enable-sysvshm –enable-sysvmsg –enable-wddx –with-kerberos –with-imap-ssl –enable-zip –with-xsl –with-tidy –enable-soap
# make && make install
============
If you got error while executing ‘make’ command like “Compile: /usr/bin/ld: cannot find -lltdl, collect2: ld returned 1 exit status” please install libtool using following command.
yum install libtool-ltdl-devel
Copy php.ini-recommended from the install directory to /opt/php52/
2. Install php 5.4
This one also going to install in separate location say /opt/php54
===========
#./configure –prefix=/opt/php54 –with-config-file-path=/opt/php54 –with-curl –enable-cli –enable-discard-path –with-mysql –with-mysqli –enable-bcmath –enable-ftp –enable-magic-quotes –with-pear –enable-sockets –with-zlib –with-gd –with-jpeg-dir=/usr/lib –with-libdir=lib64 –with-bz2 –enable-calendar –with-curl –enable-dbase –enable-exif –with-gettext –with-gmp –enable-mbstring –with-mcrypt –with-mhash –with-ncurses=shared –with-openssl –enable-pcntl –with-pdo-mysql –with-pspell –with-readline –enable-shmop –with-snmp=shared –enable-sysvsem –enable-sysvshm –enable-sysvmsg –enable-wddx –with-kerberos –with-imap-ssl –enable-zip –with-xsl –with-tidy –enable-soap
# make && make install
===========
Copy php.ini-production from install directory to /opt/php54.
Now in apache configuration file include following line or uncomment it.
Include conf/extra/httpd-fastcgi.conf
Now create this file “/usr/local/apache/conf/extra/httpd-fastcgi.conf” and add below contents to it.
==========
LoadModule fastcgi_module modules/mod_fastcgi.so
FastCgiIpcDir /opt/tmp/fcgi
AddHandler fastcgi-script .fcgi
FastCgiConfig -autoUpdate -singleThreshold 200 -killInterval 600 -idle-timeout 150
AddType application/x-httpd-php .php
ScriptAlias /fastcgi/ /usr/local/apache/cgi-bin/
<Directory “/usr/local/apache/cgi-bin”>
Options ExecCGI
SetHandler fastcgi-script
Order allow,deny
Allow from all
</Directory>
==========
Make sure that “mod_fastcgi.so” exists in the modules directory. Now create a directory “/opt/tmp/fcgi” and give 777 permission.
Navigate to directory “/usr/local/apache/cgi-bin/”.
a. Create a file for php 5.2 say “php52.fcgi” with 755 or executable permission and add following contents to it. Make sure you have added correct path for php-cgi binary.
==========
#!/bin/sh
PHP_FCGI_CHILDREN=8
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /opt/php52/bin/php-cgi
==========
Similarly create a file for php 5.4 say php54.cgi with following contents.
==========
#!/bin/sh
PHP_FCGI_CHILDREN=8
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /opt/php54/bin/php-cgi
==========
Finally go to virtual host section of each domain and specify which php to use for the domain. Imagine I have to use php 5.2 for a domain called abc.com. Go to virtual host of domain abc.com and add following line to it.
==========
Action application/x-httpd-php /fastcgi/php52.fcgi
==========
For domains using php version 5.4 add following line.
==========
Action application/x-httpd-php /fastcgi/php54.fcgi
==========
Finally restart apache service. Thats it. Put a php info page and check php versions displaying.
Thanks,
Jeen
PHP Script to download multiple files concurrently
This script will show you an example of how to download multiple files concurrently
using the curl_multi commands in PHP 5.While php itself doesn’t support multi threading and concurrency,
libcurl does and php allows us to download multiple files at the same time from php.
//using the curl_multi commands in PHP 5.While php itself doesnt support multi threading and concurrency,
//libcurl does and php allows us to download multiple files at the same time from php.// Files to download
$urls = array(‘http://abc.com/docs/cdbwpohq0ayey.pdf’,
‘http://abc.com/docs/8wyxlxfufftas.pdf’,
‘http://abc.com/docs/9q29bbglnc2gk.pdf’,);
$save_to=’/tmp/'; // Path to save files in
$mh = curl_multi_init();
// Add curl multi handles
foreach ($urls as $i => $url) {
// Path to save files in
$g=$save_to.basename($url);
$conn[$i]=curl_init($url);
$fp[$i]=fopen ($g, “wb”);
curl_setopt ($conn[$i], CURLOPT_HEADER ,0);///it should come first
curl_setopt ($conn[$i], CURLOPT_FILE, $fp[$i]);
curl_multi_add_handle ($mh,$conn[$i]);
}
// Download the files
do {
$n=curl_multi_exec($mh,$active);
}
while ($active);
foreach ($urls as $i => $url)
{
curl_multi_remove_handle($mh,$conn[$i]);
curl_close($conn[$i]);
fclose ($fp[$i]);
}
curl_multi_close($mh);
?>
Adding SSL to a site in nginx
1) First create a folder where SSL certs are to be placed
#mkdir /etc/ssl/certs
#cd /etc/ssl/certs
Create key
# openssl genrsa -out abc.com.key 2048
# chmod 600 abc.com.key
Generate CSR
# openssl req -new -key abc.com.key -out abc.com.csr
If all that you need is a self signed certificated, do as follows.
# openssl x509 -req -days 365 -in abc.com.csr -signkey abc.com.key -out abc.com.crt
Otherwise get the cert signed and place it in /etc/ssl/certs/abc.com.crt
Now, open up your nginx config file for this domain.
# vi /etc/nginx/sites-available/abc.com
If you are planning to use both http and https, you need two separate server modules in your config. Hence, copy the config for 80 and put it to the bottom of the page and make modifications. What you need to add to the https server module is given below. Paste it before server_name line.
server {
listen 1.2.1.3:443 ssl;
ssl on;
ssl_certificate /etc/ssl/certs/abc.com.crt;
ssl_certificate_key /etc/ssl/certs/abc.com.key;
server_name abc.com www.abc.com
….
…
}
Now restart nginx.
#/etc/rc.d/init.d/nginx restart
Now try accessing site with https.