Metasploit Development Environment
The Metasploit Framework is a pretty complex hunk of software, at least according to Ohloh. So, getting started with development can be daunting even for veteran exploit developers. This page attempts to demystify the process of getting your Metasploit development environment set up through submitting a "pull request" to get your exploit into the standard distribution.
This documentation assumes you're on some recent version of Ubuntu Linux. If not, then you're going to be on your own on how to get all your dependencies lined up. If you've successfully set up a development environment on something non-Ubuntu, and you'd like to share, let us know and we'll link to your tutorial from here.
Throughout this documentation, we'll be using the example user of "Fakey McFakepants," who has the e-mail address of "[email protected]" and a login username of "fakey."
The bare minimum for working on Metasploit effectively is:
apt-get -y install \
build-essential zlib1g zlib1g-dev \
libxml2 libxml2-dev libxslt-dev locate \
libcurl4-openssl-dev git-core \
libssl-dev openssl autoconf bison curl wget \
postgresql postgresql-contrib libpq-devNote that this does not include an appropriate text editor or IDE, nor does it include the Ruby interpreter itself. We'll get to that in a second.
Most (all?) standard distributions of Ruby are lacking in one regard or another. Lucky for all of us, Wayne Seguin's RVM has been getting steadily more excellent in providing several proven Ruby interpreters. Visit https://rvm.io/ to read up on it, or just trust that it'll all work out with a simple:
$ curl -L get.rvm.io | bash -s stableFollowed by
$ source ~/.rvm/scripts/rvmAnd finally:
$ rvm install 1.9.3-p125What this all does is fetch RVM, which performs a bunch of shell voodoo, and finally installs Ruby version 1.9.3 patchlevel 125 (there are lots of other Rubies to choose from, but we like this one the most right now). Assuming all goes as planned, you should end up with something like this in your shell.
Once that's all done, you can move on to setting up your preferred editor. Far be it from us to tell you what editor you use -- people get really attached to these things for some reason. Once we have some docs put together for sensible defaults for a couple of the more popular editors out there, we'll list that here.
The entire Metasploit code base is hosted here on GitHub. If you have an old Redmine account over at dev.metasploit.com, that's not going to do much for you since the switch-over. The process for creating an account is pretty simple.
None of this is exactly rocket science.
Once that's all done, you need to set up an SSH key to associate with your new GitHub identity (this step is not optional, so good on GitHub for forcing this minimal level of security).
The Metasploit core developers recommend you set up new SSH key pair to associate with GitHub, rather than reuse that same old tired key you have in 50 other authorized_keys files around the world. Why not just start fresh? It's easy and fun:
$ ssh-keygen -t -rsa -C "[email protected]"Just follow the prompts, pick a name for your key pair (I use "id_rsa.github"), set a password, and you should end up with something like:
Next, go to https://github.com/settings/ssh (which can be navigated to via Account Settings > SSH Keys), and click "Add SSH key" :
You'll be presented with a screen to copy-paste your public SSH key (not the private one!). Easiest thing to do is to cat your newly created key, select, and copy-paste it:
Once that's done, you'll have a key associated, and you'll get e-mail about it as well. Eyeball the fingerprint and make sure it all matches up.
The real moment of truth is when you test your SSH key. If you named it something funny like I did, don't forget the -i flag, use -T to avoid allocating a terminal (you won't get one anyway), and note that you are going to use literally "[email protected]" as the username (not your name or anything like that).
$ ssh -i ~/.ssh/id_rsa.github -T [email protected]Your console should look like:
I hate having to remember usernames for anything anymore, so I've gotten in the habit of creating Host entries for lots of things in my ~/.ssh/config file. You should try it, it's fun and it can shorten most of your ssh logins to two words.
For the rest of these instructions, I'm going to assume you have something like this in yours:
Host github
Hostname github.com
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_rsa.github
To check that it works, just ssh -T github, and your result should be just like this:
Finally, you're ready to set up your local git config file, if you haven't already:
git config --global user.name "Fakey McFakepants"
git config --global user.email "[email protected]"Cat your ~/.gitconfig to ensure you have at least that set (and remember, your e-mail address needs to match the address you set back when you ssh-keygen'ed):
The rest of this document will walk through the usual use case of working with Git and GitHub to get a local source checkout, commit something new, and get it submitted to be part of the Metasploit Framework distribution. The example here will commit the file 2.txt to test/git/ , but imagine that we're committing some new module like ms_12_020_code_exec.rb to modules/exploits/windows/rdp/.
Now that you have a GitHub account, it's time to fork the Metasploit Framework. First, go to https://github.com/rapid7/metasploit-framework , and click the button:
Hang out for a few seconds, and behold the animated "Hardcore Forking Action:"
Once that's done, switch back over to your terminal, make a subdirectory for your git clones, and use your previously defined .ssh/config alias to clone up a copy of Metasploit:
$ mkdir git
$ cd git
$ git clone github:mcfakepants/metasploit-framework.gitYou should end up with a complete copy of Metasploit in the metasploit-framework subdirectory, like so:
Now might be a good time to decorate up your prompt. I've hacked this gist together for my ~/.bash_aliases. It's a little ugly, but it works.
What this does is let me know on the command line prompt which version of Ruby, which gemset, and which Git branch I happen to be in. The end result looks like this:
Now that you have a source checkout of Metasploit, and you have all your prerequisite components from apt and rvm, you should be able to run it straight from your git clone with ./msfconsole -L:
One of the main reasons to use Git and GitHub is this whole idea of branching in order to keep all the code changes straight. In other source control management systems, branching quickly becomes a nightmare, but in Git, branching happens all the time.
You start off with your first branch, "master," which you pretty much never work in. That branch's job is to keep in sync with everyone else. In the case of Metasploit, "everyone else" is rapid7/metasploit-framework/branches/master. Let's see how you can keep up with the upstream changes via regular rebasing from upstream's master branch to your master branch.
This is pretty straightforward. From your local branch on the command line, you can:
$ git remote add upstream git://github.com/rapid7/metasploit-framework.git
$ git checkout upstream/masterThis lets you peek in on upstream, after giving a warning about being in the "detatched HEAD" state (don't worry about that now). From here you can do things like read the change log:
$ git log --pretty=oneline --name-only -3It should all look like this in your command window:
It's pretty handy to have this checkout be persistent, though, so you can reference it later. So, type this:
$ git checkout -b upstream-masterAnd this will create a new branch called "upstream-master." Now, switch back to your master branch and fetch anything new from there:
$ git checkout master
$ git fetchAnd finally, rebase against the upstream. Rebasing is the easiest way to make sure that your master branch is identical to the upstream master branch. If you have any local changes, those are "rewound," all the remote changes get laid down, and then your changes get reapplied. It should all look like this:
Of course, you might occasionally run into rebase conflicts, but let's just assume you won't for now. :) See the "Other links" for dealing with merge and rebase conflicts.
A note on terminology: In Git, we often refer to "origin" and "master," which can be confusing. "Origin" is a remote repository which contains all of your branches. "Master" is a branch of the source code -- usually the first branch, and the branch you don't tend to commit directly to.
"Origin" isn't Rapid7's repository -- we usually referred to that repo as "Upstream." In other words, "upstream" is just another way of referring to the "rapid7" remote.
Got it? "Origin" is your repo up at GitHub, "upstream" is Rapid7's GitHub repo, and "master" is the primary branch of their respective repos.
All right, moving on.
Any time you rebase from upstream, you're likely to bring in new changes, since we're committing stuff all the time. This means that when you rebase, your local branch will be ahead of your remote branch. To get your remote up to speed, just
$ git push origin masterIt should all look something like this:
Switch back to your browser, refresh, and you should see the new changes reflected in your repo immediately (those GitHub guys are super fast):
Finally, let's get to pull requests. That's why you're reading all this, after all. This is the basic procedure that @corelanc0d3r suggested for Metasploit contributors. Thanks to @corelanc0d3r for initially writing this all down from a contributor's perspective.
First, create a new branch:
git checkout -b module-ms12-020Write the module, putting it in the proper subdirectory. Once it's all done and tested, add the module to your repo and push it up to origin:
git add <path to new module>
git commit -m "added MS012-020 RCE for Win2008 R2"
git push origin module-ms12-020That command set should look something like this:
In your browser, go to your newly created branch, and click the Pull Request.
This will automatically reference upstream's master as the branch to land your pull request, and give you an opportunity to talk about how great your module is, what it does, how to test it, etc.
Once you click Send Pull Request, you'll be on upstream's pull queue (in this case, mcfakepants has created pull request #356, which is one of 17 open pull requests).
Depending on the position of the stars, someone from the Metasploit core development team will review your pull request, and land it, like so:
Now, keep in mind that actually landing a pull request is a little more involved than just taking your commit and applying it directly to the tree. Usually, there are a few changes to be made, sometimes there's some back and forth on the pull request to see if some technique works better, etc. To have the best chances of actually getting your work merged, you would be wise to consult the [Acceptance Guidelines]
The upshot is, what's committed to Metasploit is rarely exactly what you initially sent, so once the change is committed, you'll want to rebase your checkout against master to pick up all the changes. If you've been developing in a branch (as you should), you shouldn't hit any conflicts with that.
Now that everything's committed and you're rebased, if you'd like to clean out your development branches, you can just do the following:
$ git branch -D module-ms12-020
$ git push origin :module-ms12-020Note that Git branches are cheap (nearly free, in terms of disk space), so this shouldn't happen too terribly often.
First off, thanks to @corelanc0d3r for articulating much of this. If you have suggestions for this wiki, please let @todb-r7 , @wchen-r7 , or @jlee-r7 know.
This document should be enough to get your Metasploit development career started, but it doesn't address huge areas of Git source control management. For that, you'll want to look at the Git Community Book, the many answered questions on StackOverflow, and the git cheat sheet.