I have tried a few ways to upgrade, including removing the current install, and then using apt-get to install the newest version, but the Ubuntu repositories are sometimes out of date, and also a few things have gone wrong.
Here is the method I prefer which involves manually downloading, unzipping and then copying the new binaries over the existing ones:
1. Go to your temp directory : cd /tmp
2. Download the latest package from the MongoDB website to /tmp : sudo wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.4.1.tgz
3. Unzip the package: sudo tar -zxvf mongodb-linux-x86_64-2.4.1.tgz
4. Stop MongoDb – sudo service mongodb stop
5. Go to the folder where you just unzipped the files: cd /tmp/mongodb-linux-x86_64-2.4.1/bin
6. Copy the new files and overwrite the existing files: sudo cp -i * /usr/bin
7. Start the mongo service: sudo service mongodb start
8. Check that MongoDB is running and your MongoDB version : mongo