Start with Git Large File Storage

Git Large File Storage (LFS) is a Git extension for use with projects with lots of large binary files such as photos, graphics, CAD, and other non-clear-text files. It helps by storing your large binary files outside the Git repository to keep the repository size manageable.

This feature is particularly helpful for open hardware projects where design source files are often large binary files. Check out Git for Open Hardware for more info on using Git for your Open Hardware projects.

Setting up Git LFS is quick and requires little extra maintenance or learning from contributors. It also helps that it is supported by a Git hosting service you are likely already using including: Github, GitLab, Gitea, Phabricator, and Bitbucket.

This guide assumes you have already used Git a bit. If not, check out Learn Git in 20 Minutes video, github.io resources, and Codecademy course to list a few to get you started.

Installing git-lfs

GNU/Linux

Git LFS is available in the recent releases of most Linux distros including Fedora, Pop!_OS, Ubuntu, CentOS, and the coming Debian 10. For those distros, installing git-lfs is quick:

Fedora/CentOS

sudo dnf install git-lfs

Ubuntu/Pop!_OS/Debian

sudo apt install git-lfs

Git LFS also provides binaries for additional distros and older versions through packagecloud.io and the project’s Github Releases page.

MacOS

The quickest way to install for MacOS is through Homebrew. Once you have Homebrew installed run:

brew install git-lfs

Windows

Git LFS provides a Windows installer for git-lfs. You can find the most recent installer on the project Github Releases page.

Set up a Git repository for Git LFS

The beauty of Git LFS is that you can quickly set up a project for LFS from the start and not have to worry about it again. The git-lfs extension allows you to track which file type suffixes should be stored in Git LFS rather than the repository. The configuration of which file suffixes to track is also stored in the repository and so shared with all repository collaborators.

If you are starting with a fresh project first create the Git repository:

mkdir My_Project && cd My_Project
git init

Initialize LFS for the repo with:

git lfs install

At this point you can add tracking for any suffix patterns you would like to track with LFS.

git lfs track '*.jpg'

This will track any file with the suffix of ‘.jpg’. You can also list multiple suffix patterns in one command:

git lfs track '*.pdf' '*.png'

Using track without a file suffix pattern will return a list of the patterns currently being tracked.

git lfs track

returns:

Listing tracked patterns
*.jpg (.gitattributes)
*.png (.gitattributes)
*.pdf (.gitattributes)

You should also be aware that the suffix pattern can be case sensitive so it is best to include both cases:

git track '*.jpg' '*.JPG'

Now if you check git status for changes you will see there is a new file, .gitattributes. This file contains the LFS tracking pattern configuration. You should now commit this file to your repository and push to your remote Git host to ensure other collaborators will also push LFS tracked files to the LFS rather than the repository.

If you now commit a .jpg, .png, or .pdf suffixed file to the repository only the current version of the file will be stored in the repository. All other versions of the file from other commits will be stored in LFS.

One warning to be aware of with Git LFS is that it will be required to clone repositories that have LFS initialized. Otherwise, if the user attempts to clone the repository without git-lfs installed, Git will return an error and fail the clone. It would be best to include in your project README that the repository requires git-lfs and point to instructions to install it before attempting to clone the repository.

Using Git LFS in an existing repository

It is easiest to setup LFS with a new project; however, you can use LFS on an existing repository but with a bit of extra work. The setup is the same as above except after you add the file suffixes to track and commit the .gitattributes file you need to remove the file pattern suffix from the git cache and recommit:

git rm --cached *.jpg
git add *.jpg
git commit -m "Convert *.jpg to LFS"

From that commit forward any commits with a .jpg suffix will be stored in LFS.

Warning, when converting an existing repository to LFS, all previous commits prior to initializing LFS tracked suffixes will still be stored in the Git history. The only way to clear out those old versions of the files from the Git history would be to rewrite the history of the repository and perform a force push to your remote.

Unless you are the only contributor on a repository, you should not do this. It would cause havoc for your other contributors as their local repositories would no longer be able to push or pull from the remote and would require they re-clone a fresh local repository. It would also break any issue tracking you had previously connected to commits that are rebased.

If you understand the consequences and still would like to rebase your earlier commits to remove the Git tracked files check out the git-lfs guide on migrating data and a lengthy issue ticket about migrating existing data to LFS.

Have any experiences with Git LFS to add, corrections, suggestions? @stabadie or dm me on Twitter.