Citations with Citeulike + Bibtex

I’ve started writing my senior thesis, so I’ve started looking for a way to manage all my sources. My thesis is being written in LaTeX, so using BibTeX was an obvious choice. BibTeX uses a .bib file to hold information about all your sources: see here for more details. You can populate this .bib file by hand, but it’s easier to use a citation manager. I went with Citeulike.

You can give Citeulike a link to a journal article, a DOI, or a Pubmed ID, and it will automatically gather relevant information. You can export your citations to a .bib file, which is compatible with LaTeX. Even better, you can automate this retrieval process, since Citeulike allows you to access your citations using URLs, which can be processed with tools like wget or curl.

I wrote a Python script (link) to automate fetching a BibTeX file for all your citations, based off these wget commands.

The script uses your Citeulike username and password to log in and then download the .bib file. You need to log in, otherwise you would only be able to access citation entries you made “public.” The URL for accessing the .bib file is

http://www.citeulike.org/bibtex/user/USERNAME

where USERNAME is your username.

Additionally, Citeulike allows you to tag your citation entries. If you use Citeulike for multiple projects, you can have a tag for each project and download the .bib file for the project with tag TAG using the following URL:

http://www.citeulike.org/bibtex/user/USERNAME/tag/TAG

My script supports both these functions. The -o and -t arguments are optional but can be used to change the output filename and what tag to use.

usage: download_citations.py [-h] [-o OUTPUT] [-t TAG]

Download bibtex citation file from citeulike.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Bibtex output file name. Default: export.bib
  -t TAG, --tag TAG     Which tag to use. By default, no tags are used.

You’ll need to set the USERNAME and PASSWORD variables to your own Citeulike username and password, respectively, before continuing. Using this script you can automate fetching those bibliography files and make sure your citations are up to date. Enjoy!

Caveat: Citeulike’s security certificate expired on June 8, 2015 and hasn’t been renewed since, so I am using non-secure HTTP, not HTTPS, to log in. It’s probably best if you use a different password for Citeulike so that people can’t steal your Facebook/email/bank passwords.

#!/usr/bin/python

##########################################
## Downloads bibtex file from Citeulike.
## By: Andy Shi
## For help and usage, execute
##  python download_citations.py -h
## Inspired by:
## http://linuxtoosx.blogspot.com/2012/10/downloadbackup-citeulike-library.html
##########################################

import argparse
import requests

# set your Citeulike username and password here
USERNAME = "USERNAME"
PASSWORD = "PASSWORD"

# argparse configuration
DEFAULT_OUTPUT = "export.bib"
DESCRIPTION = "Download bibtex citation file from citeulike."

# process command line arguments
def handle_cli():
    parser = argparse.ArgumentParser(description = DESCRIPTION)
    parser.add_argument("-o", "--output", default = DEFAULT_OUTPUT,
            help = "Bibtex output file name. Default: " + DEFAULT_OUTPUT)
    parser.add_argument("-t", "--tag",
            help = "Which tag to use. By default, no tags are used.")
    args = parser.parse_args()
    return(args)

def main():
    args = handle_cli()
    with requests.Session() as s:
        # log in to citeulike
        payload = {'username': USERNAME, 'password': PASSWORD, 'perm': 1}
        r_login = s.post("http://www.citeulike.org/login.do", data=payload)

        if args.tag:
            # download citations with a certain tag
            url = \
            "http://www.citeulike.org/bibtex/user/{}/tag/{}".format(USERNAME,
                    args.tag)
        else:
            # download all citations
            url = "http://www.citeulike.org/bibtex/user/{}".format(USERNAME)
        r_bibtex = s.get(url)

        # write bibtex to file
        chunk_size = 1024
        with open(args.output, 'wb') as fd:
            for chunk in r_bibtex.iter_content(chunk_size):
                fd.write(chunk)

if __name__ == "__main__":
    main()