Removing unwanted files from your build

2008 November 14

We use Subversion (SVN) to manage our product, because we have a couple developers working on the same code, and it makes sense to have versioning control when more than 1 developer is working on a project.  I personally use it even when I’m the only dev, because I make mistakes and want to go back in time.  Also, SVN gives you one more backup location for your files.

Also, we use Macs and one of us uses Dreamweaver.

Why does this matter?  Because it means we have a TON of extra files sitting in our development directories.  We have .svn, .DS_Store, dwsync.xml, and _notes directories.  If we had a windows person, our images directories would be littered with Thumbs.db files.

There are a couple ways to remove your unwanted files, I will go over some of them here.

1. Search and Delete

This is time consuming and a pain.  Don’t do this, it will take you forever and depending on how good you are, you’ll probably miss some files.  To say the least, its frustrating.

2. svn export

You can use this command line tool, to export your entire project from svn, without any of the .svn files.  If you keep the other files mentioned out of your repo, then you are as good as gold and ready to go.  To use this command just type:

$ svn export http://path.tld/to/your/repo/folder path/to/export/location
  1. You will of course need to replace the paths with correct paths for you, but you get the point.
  2.  
  3. <strong>3. Transmit</strong>
  4.  
  5. If you have a mac, and you have transmit you can use it to scrub your files.
  6.  
  7. The synchronize tool in Transmit allows you to synchronize your local directory with a server directory, and remove files as you go.
  8.  
  9. Steps:
  10. <p style="padding-left: 30px;">1) Log into your server</p>
  11. <p style="padding-left: 30px;">2) Make sure "Your stuff" is pointed at the directory you want to upload</p>
  12. <p style="padding-left: 30px;">3) Make sure "Their stuff" is pointed at the destination directory</p>
  13. <p style="padding-left: 30px;">4) Hit the synchronize button at the top</p>
  14. <p style="padding-left: 30px;">5) In the drop down, click the "Skip items with specified names" and then click the button</p>
  15. <p style="padding-left: 30px;">6) Add in all the file extensions you want to ignore (.svn, _notes, .DS_Store, Thumbs.db, etc)</p>
  16. <p style="padding-left: 30px;">7) Click the "Simulation Only" button.  This will show you what the end result will be, without transferring the actual files.</p>
  17. <p style="padding-left: 30px;">8) You should get a report option in the results screen, it will look something like this:</p>
  18.  
  19. <div class="mceTemp" style="padding-left: 30px;"><dl id="attachment_100" class="wp-caption alignnone" style="width: 310px;"> <dt class="wp-caption-dt"><a href="http://devblog.cloudsync.com/wp-content/uploads/2008/11/transmitsync.jpg"><img class="size-medium wp-image-100" title="Transmit Sync Results" src="http://devblog.cloudsync.com/wp-content/uploads/2008/11/transmitsync-300x290.jpg" alt="Transmit Sync Results" width="300" height="290" /></a></dt> <dd class="wp-caption-dd">Transmit Sync Results</dd> </dl></div>
  20. <p style="padding-left: 30px;">The grey minus sign indicates that the files were ignored.</p>
  21. <p style="padding-left: 30px;">9) If the report is acceptible, you can go back to synchronize.  All your settings will be saved, and you can run it without the "simulation" option.  This will actually push the files, so make sure you are ready.</p>
  22.  
  23. A side note, if you want there is also an option to remove all abandoned files on the server.  This will get rid of any of the files you DID NOT transfer.  A word of caution about this, if you have scripts in the server directroy you want to keep, don't use this option.  Once server files are deleted, unless you have good backups, they are gone forever.
  24. <h3>4. rsync</h3>
  25. This is one of the most powerful tools on a *nix box.  If you don't have it, install it. You will absolutely love it. Rsync allows you to "synchronize" two directories together. This can be two local directories, or a local and a server directory.
  26.  
  27. There are specific directives/options for rsync that allow you to exclude files.  If you want more information that I am giving you here, use the manual page by typing the following into your Terminal window:
  28. <pre lang="bash">$ man rsync

The rsync options I will be using are the following:

-a: archive mode, this is the same as -rlptgoD.  Gets files recursively, copies symlinks as symlinks, preserves permissions, preserves times on files, preserves the group, preserves the owner, and preserves device files and special files.

-v: verbose, shows you what rsync is doing

-z: compresses the files during transfer, which makes the entire process run much faster

-n: Dry run, use this the first time to see what it will look like when it is actually run.  This way you do not accidentally remove something you need when pushing file to the server

--cvs-exclude: Yes, it says CVS, but it will also exclude .svn files and directories

--exclude-from: allows you to specify a file, which will have types of files to ignore in it.  The suggested file to use is ~/.rsync/exclude and it should look something like this:

$ cat ~/.rsync/exclude
  1. .DS_Store
  2. dwsync.xml
  3. Thumbs.db
  4. _notes
  5. cache/*

You can choose to remove all files with a specific name, or files in a directory (like "cache/*").

So my final script for rsync looks like this:

rsync -avz --cvs-exclude --exclude-from=~/.rsync/exclude /starting/path/for/files /end/location/for/files

Resources:

Ignore svn files with Transmit

Rsync Tips and Tricks

Subversion Documentation

Sphere: Related Content

No Comments

Leave A Comment

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS