Skip to content

SysAdmin

Jump to bottom Edit New page
Leo Lapworth edited this page Aug 12, 2013 · 59 revisions

SysAdmin FAQ

Manual maintenance issues

How to reindex a missing module?

cd api.metacpan.org
bin/metacpan release http://cpan.metacpan.org/authors/id/X/XS/XSAWYERX/MetaCPAN-API-0.33.tar.gz --latest

How to index all unindexed dists uploaded over the last 24 hours?

bin/metacpan release --skip --age 24 --latest ~/CPAN/authors/id/

How to index the latest Perl release

bin/metacpan release http://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.16.0.tar.bz2
bin/metacpan release --status latest http://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.16.1.tar.bz2

The above syntax will force the status bit to "latest", which we need to do manually only for new, latest Perl releases. Make sure you reindex the predecessor as well.

Restarting services

The following services are set up:

metacpan-www
metacpan-api
metacpan-rrr
metacpan-watcher
elasticsearch

Each of those services can be restarted by calling service $name restart (as superuser).

How to deploy a new version of MetaCPAN-API and MetaCPAN-Web?

Log in as the metacpan user (which loads perlbrew automatically), go to the appropriate folder (~/metacpan.org or ~/api.metacpan.org), pull from github and restart the service as root (rcmetacpan-ww restart or rcmetacpan-api restart).

For now prereqs can be installed manually with sudo /home/metacpan/bin/install_modules Foo::Bar.

How to increase storage space for ElasticSearch and the CPAN mirror?

The CPAN mirror and the ElasticSearch data are stored in /var/cpan and /var/elasticsearch, respectively. Those are filesystems on top of the LVM LVs /dev/mapper/vg0-cpan and /dev/mapper/vg0-elasticsearch.

To increase the space available on one of them, change the following example, which adds an additional 100 MB for the CPAN mirror. There's no need to unmount anything.

# Show current usage (and what is free, see note below)
pvscan
# Grow the LVM volume
lvextend -L +100M /dev/mapper/vg0-cpan
# Extend the filesystem to the fit the new LV size
resize2fs /dev/mapper/vg0-cpan

Do NOT allocate all the unused space to logical volumes. We need some free space to use by LVM snapshots during the backup process. We haven't actually checked how much spare space we need for that, so let's play it safe and say that at least 1.5 GiB should be left alone.

Need to clean out /var/tmp/metacpan/ ?

find /var/tmp/metacpan/source/ -maxdepth 2 -type d -mtime +215 | head -5000 | xargs sudo rm -rf

This doesn't solve it as such - but cleans up files that haven't been modified/extracted in a long while

Network/high level issues

Managing DNS

We are currently sponsored by Dyn with a DynECT Managed DNS Lite account.

URL: https://manage.dynect.net

When delegating your domain names, please use the following nameservers:

ns1.p24.dynect.net

ns2.p24.dynect.net

ns3.p24.dynect.net

ns4.p24.dynect.net

The best place to get started is look at the DynECT Managed DNS Lite User Manual located at: https://manage.dynect.net/help

Our contact at Dyn is Chris Gonyea [email protected]. Also, alh in #metacpan works for Dyn and can help with technical issues.

ByteMark mirror

Specs:

What is the procedure if the server is unreachable?

ByteMark

We should be able to fix most stuff because we have console access (see above)

Booking.com

Contact the booking staff either by email ([email protected]) or in emergencies by phone (+31207153409). Most problems are better solved on IRC. Our contact on irc.perl.org is Seveas.

Where are log files stored?

ElasticSearch logs can be found in /opt/elasticsearch-0.20.2/logs

Where are system monitoring reports stored?

The ElasticSearch status can be queried from within the box:

$ curl localhost:9200/cpan/_status?pretty

$ curl localhost:9200/_cluster/health/cpan_v1?level=shards

http://munin.bm-n2.metacpan.org/metacpan.org/bm-n2.metacpan.org/ http://nagios.omega.pqpq.de/

web api watcher

Where are backups stored

Add a custom footer

Meta

Clone this wiki locally