Setting up Zotero with WebDAV

After skipping around many a reference manager I settled on Zotero for my academic needs. While Zotero is great (and not owned by Elsevier like a certain other reference manager) it comes with a storage size limit if you want to sync files. Unless, that is, you choose to use WebDAV for your file storage needs. Unfortunately, but understandably, Zotero does not come with a guide to setting up WebDAV—there are simply too many ways to go about it—so this guide, written primarily for my future self but open to anyone who might need it, should help set up a WebDAV server and establish file syncing with Zotero.

Before we begin I shall address the elephant in the room, but feel free to skip ahead to the prerequisites if you do not need this explanation. Why not just use bibTeX? I do use bibTeX (more like bibLaTeX really) for documents I share exclusively as PDFs but we often end up working with collaborators, supervisors and publishers who swear by Microsoft Word, and Zotero can be handy in such times. Even otherwise, when I am not worshipping at the alter of LaTeX, I think Zotero is a fantastic program that works well with bibLaTeX, allowing folder-based, on-the-fly creation of .bib files for use with LaTeX and other programs. That can be the best of both worlds: use Zotero to maintain a reference library with a wonderful GUI, keeping your files, notes and everything else related to your reference library in sync, while also outputting .bib files for when you might need them.

A program like Zotero, in my opinion, is much more useful when you can use it to manage all your related content viz. PDFs, highlights, annotations, notes etc. I have even started tagging things better now that I use Zotero to the fullest rather than simply as a place to tabulate papers I have read and look to return to.

Prerequisites

There are a few prerequisite functionality that need to be in place before we start setting up WebDAV. First, I assume you have an Apache2 server (not only because this is what I use myself but also because this guide is for setting up the WebDAV built into Apache2). You will also need Certbot, basic Terminal skills, and at least a passing familiarity with managing a server. I run a Debian server so the commands in this guide work on Debian or Ubuntu machines.

This guide is also for setting things up on your own server (a VPS say) so a server is a prerequisite. I will also assume you already have Zotero installed on your system and logged in, with a few (or many) references and files saved, regardless of whether or not you already have file sync set up.

Next, I assume you have either direct root access or root access via a superuser (the latter is preferred for safety). The commands below omit the sudo call that you would need for non-root users, so make sure to prepend commands with sudo as necessary and that you know your superuser password.

Lastly, I assume you plan to access your files over a subdomain like sub.domain.tld which you have set up on your DNS as well. You do not need a separate folder, user etc., just a proper DNS route to your server.

Step 1

Set up a virtual host

Head to the standard Apache2 sites-available folder and create a configuration file for your sub.domain.tld as follows:

cd /etc/apache2/sites-available/ && nano sub.domain.tld.conf

Fill it up with the appropriate content. You can use my file below with two changes: (a) enter your IP address where it says ##IP## and (b) enter your subdomain where it says sub.domain.tld. Also keep in mind that this only addresses access via a proper SSL certificate, which is a good thing because you do not want your WebDAV server unsecure.

Alternately, you could use your own file so long as you specify the <Directory>...</Directory> sections appropriately as given below.

<VirtualHost ##IP##:443>
    ServerName sub.domain.tld
    ServerAdmin webmaster@domain.tld

    DocumentRoot /var/www/webdav
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/webdav/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        Alias /zotero /var/www/webdav/zotero
        <Location /zotero>
                DAV On
                AuthType Basic
                AuthName "webdav"
                AuthUserFile /usr/local/webdavusers/webdav.passwords
                Require valid-user
        </Location>

    SSLEngine On

    # Reverse Proxy
    SSLProxyEngine Off
    SSLProxyVerify none
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/sub.domain.tld/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/sub.domain.tld/privkey.pem
</VirtualHost>

At this point run Certbot to set up SSL certificates for your subdomain. This is critical because one cannot be encouraged to run a WebDAV server without proper security.

Step 2

Set up your WebDAV directory

Notice how in the configuration file above we specified a /var/www/webdav path? That does not exist yet so we need to create it with—

mkdir /var/www/webdav

And to ensure things are working, set up a dummy index.html file and set up the correct permissions:

sh -c 'echo "Hello, world!" > /var/www/webdav/index.html' && chown www-data:www-data /var/www/webdav

This index.html file is temporary. You can do whatever you want with it later. We just need it now to ensure things are functioning. To check, refresh your Apache2 server first:

a2ensite sub.domain.tld && service apache2 reload

This will enable the new subdomain on Apache2 and restarting the server will have it functioning as expected. Visit sub.domain.tld and check that you have a “Hello, world!” page showing.

Step 3

Enable WebDAV

So far all we have done is set up your subdomain to access a non-existent WebDAV server. Let us therefore set up the WebDAV server itself. Helpfully, this is built into Apache2 by default but comes disabled out of the box. Start with—

a2enmod dav_fs

This turns on the Apache2 WebDAV switch if you will, but there are still a couple of other things to set up, such as a folder for Zotero to use. Keeping consistency with the configuration file above I call this folder zotero rather unimaginatively. If you choose to call it something else, make sure to also change the name in your configuration file. However, remember that in Zotero settings the program necessarily requires files to ultimately rest in a folder called ‘zotero’; so if you set a different folder name at this stage, you will still need to create a subfolder called ‘zotero’ inside it.

mkdir /var/www/webdav/zotero && chown www-data:www-data /var/www/webdav/zotero && service apache2 restart

You could omit the last command but I like to restart Apache2 at this point for good measure. Accessing sub.domain.tld/zotero should now be possible (usually you can expect to see the contents of a blank folder with a link to the parent directory i.e. the subdomain root you set up previously).

Step 4

Set up a WebDAV user

If you go to Zotero settings → Sync and turn on ‘Sync attachment files in My Library using WebDAV’ you will see it needs three parameters: a URL, a username and a password. We have the URL ready now but not a username or password, so let us prepare that next:

mkdir /usr/local/webdavusers

And set up a username (remember to change it in the command below) and password using—

htpasswd -c /usr/local/webdavusers/webdav.passwords username

This will prompt you for a password. Enter one of your choice and remember it because you will need it to set up Zotero. Then restart Apache2 again.

service apache2 restart

I am quite sure you can skip at least one restart I have done so far but that is not this one. And if you get an error when you try to restart Apache2, check your configuration file and, once you fix any errors, start Apache2 using systemctl start apache2 and you should get no errors thereafter.

Step 5

Mount and secure WebDAV

In the final step we shall mount our WebDAV server as a system volume before giving Zotero a test run. Turn on davfs2 with—

apt install davfs2

This will give you a classic Debian/Ubuntu boot-style window with two options and the following message:

The file /sbin/mount.davfs must have the SUID bit set if you want to allow unprivileged (non-root) users to mount WebDAV resources.

If you do not choose this option, only root will be allowed to mount WebDAV resources. This can later be changed by running ‘dpkg-reconfigure davfs2’.

Should unprivileged users be allowed to mount WebDAV resources?

I recommend picking no but you can pick yes now and change it later using the command dpkg-reconfigure davfs2 if needed. I see no reason to let non-root users mount WebDAV so I continue to recommend choosing no at this stage.

Once you make a choice, you will find yourself back at the terminal prompt, ready to mount your zotero folder from earlier. Change your sub.domain.tld value and the folder name if you changed it from zotero at any point earlier.

mkdir /mnt/webdav && mount.davfs https://sub.domain.tld/zotero /mnt/webdav/

Provide the username and password you picked in step 4 and hit return/enter to proceed. As with the rest of this guide, these steps produce no output, which is a good thing. We just do not want any error messages.

Ensure you can access the mounted folder:

cd /mnt/webdav/ && ls

You should see the same blank contents as the WebDAV directory you set up at /var/www/webdav/zotero/ in step 2. You will see a lost+found directory which you can ignore.

Set up Zotero

Finally, you can think of setting up your Zotero. Head to Zotero settings → Sync and turn on ‘Sync attachment files in My Library using…’ and choose WebDAV from the dropdown next to it. Enter your sub.domain.tld as the URL along with your username and password from step 4.

Click ‘Verify Server’ and you should see a message that says ‘Server configuration verified: File sync is successfully set up.’ Dismiss it happily.

If at this point you get an error, check that you have permissions set up properly. You can run the following command (changing your folder name as necessary) just to be doubly sure:

chown www-data:www-data /var/www/webdav/zotero

In any case, error or not, try accessing your sub.domain.tld/zotero via your browser. You should be prompted for a username and password with a rudimentary (read, browser default) login form. This ensures not everyone can access your file storage.

Any other error, unfortunately, can be hard to debug. Such is the nature of WebDAV but once set up it should run smoothly.

Moving or switching sync services

If you already have sync set up and are moving to WebDAV, or if you are moving WebDAV servers, you may need to complete a couple of extra steps.

To switch sync services from, say, local to WebDAV or Zotero’s proprietary services to WebDAV, set things up as described above and then head to Zotero settings → Sync → Show reset options. Here, toggle on the radio button for ‘Reset File Sync History’ and click ‘Reset…’, then return to your Zotero library and force a sync either by clicking the sync button on the top-right of the Zotero window or by modifying an entry (try deleting and re-adding an attachment).

To move WebDAV servers or change folders simply copy all contents from the old folder to the new one (you should have a bunch of .prop files each with an corresponding .zip file), then set up Zotero sync via WebDAV with the new credentials and/or URL and you should be good to go.

Self-hosting files can be cheaper than the proprietary Zotero storage option but the price you pay for going that route is following a guide such as this. Some might consider that worthwhile. As of this writing Zotero storage costs about £4/month for 6GB while a VPS storage volume might cost as little as £0.5/month for 10GB, or 12.5x savings. However, if you use more than 150GB of file storage you might end up saving more with Zotero’s ‘unlimited’ storage tier, which costs about £8 right now.

In any case, enjoy your newfound academic reference library freedom, using Zotero to its fullest while keeping your files securely and privately on your own server.