IgnorantGuru's Blog

Linux software, news, and tips

Script: kscrubber

«Downloads

Download Links:
Script: downloadbrowseauthenticateinstructions
Debian/Ubuntu: packagesPPA
Description: Cleans logs and removes recent activity information from system and user files
Recommended For: Kubuntu; Arch Linux with KDE4
Tested On: Kubuntu Karmic with KDE 4.3 desktop; Arch Linux with KDE 4.4 apps
Requires: secure-delete sqlite3
License: GNU GPL v3     * SEE DISCLAIMER *
Related: sandfox
Feedback: commentsissuesKubuntu Forum Thread

Overview

Primarily designed to be run on Ubuntu and Arch with a focus on KDE4, kscrubber is a tool for removing and cleaning specific system files, logs, and user files, for the purposes of freeing disk space and also removing recent activity information. (kscrubber may work on other distros as well – just test to be sure nothing is missed due to different file locations.)

KDE4 is the most log-intensive version of KDE to date. It keeps track of every file you open, copy, move, or delete and stores this information, for no discernable reason, long-term in various temporary and configuration files, not only in the user’s home folder, but also in system folders. At times it is useful to be able to clean a system of such activity information without having to delete the entire user folder or reinstall the system – that is the purpose of kscrubber.

Also, Firefox handles your private information very poorly. For one, even when you delete your browsing history, web form history, recently visited links, and other information within Firefox, much of this information may remain in the sqlite database files, marked as invalid data, but still readable with a simple text editor (eg Kate). Second, Firefox makes no attempt to delete Adobe Flash ‘cookies’ (aka “Local Shared Objects”) [see Wired article: You Deleted Your Cookies? Think Again] and [Flash Cookies: The Silent Privacy Killer]. “This type of cookie exists on 98% of global computers, across all operating systems.” These Flash ‘cookies’ can and often are used by tracking websites to recreate conventional cookies you’ve deleted. Firefox does nothing about this. kscrubber removes them.

USE AT YOUR OWN RISK – By design, kscrubber is a fairly heavy-handed script that removes and modifies a large number of files belonging to the system, KDE, and users. While every attempt is made to avoid breaking software or causing system instability, these outcomes are not impossible. In the unlikely event any instability is encountered, a reboot should correct it, as many of the files that kscrubber removes are temporary files which will be automatically recreated by the system or software. Regardless, it never hurts to have a recent backup of your /home folder and your system. See How To Backup Operating Systems for help. In addition, kscrubber includes a built-in option to create backup copies of files it modifies (see below). SEE DISCLAIMER

You can use kscrubber as-is or modify it for your own purposes – a text editor is all that’s required.

kscrubber --help

Cleans logs and removes recent activity information from system and
user files.  This version is designed for use on Ubuntu and Arch with
a focus on KDE
Requires: secure-delete sqlite3
Must be run as root
Usage: sudo /PATH-TO-SCRIPTS/kscrubber OPTIONS
Options:
--clean                        Clean system and user files
--keepsystem                   Don't clean any system files in /var/log,
                                 /var/crash, or /tmp
--keepsystemlogs               Don't clean system logs in /var/log
--keepuser                     Don't alter user files in /home, /root,
                                 /var/tmp, or /tmp
--sim                          Simulate only (also enables --verbose)
                                 No files will be modified.
                                 Note: If used with --backup, backup
                                 files WILL be created
--force                        Don't halt for any warnings
--verbose                      Detailed feedback
--logfile FILE                 Also append output messages to FILE
                                 (place option first for best results)
--backup FOLDER                Create backup copies in FOLDER of files
                                 to be removed or edited.  Note: If used
                                 with --sim, backup files WILL be created
--onepass                      Use less secure one-pass wiping of files
                                 (faster and may be better for SSDs)
                                 Default is two-pass
--fullpass                     Use srm default (38-pass) wiping of files
                                 (more secure but much slower)
--check PATH "SEARCHSTRING"    Search recursively in PATH for files or
                                 filenames that contain SEARCHSTRING
                                 (text or a regular expression).  You may
                                 include multiple --check options to
                                 search for multiple strings.

The best way to run kscrubber is before you login to KDE. At the login screen, press Ctrl-Alt-F1 to get a console, then login and run kscrubber. When run without KDE running, kscrubber will remove most of the /var/tmp/kdecache-*/ folders for all users. (It cannot do this while KDE is running without creating system instability.) Then press Ctrl-Alt-F7 to login to KDE. When KDE starts, it will recreate these folders.

If KDE is running, kscrubber will remove a few files from /var/tmp, but it can’t remove the plasma and kpc caches, and other data. This data may contain recent activity and other private user information, including information on recently visited sites in Konqueror.

To clean your system: each user should first clear the recent browsing history within Firefox and other browsers. Next, close as many running programs as possible, and run:

sudo kscrubber --clean

You may also use the –keepsystem, –keepsystemlogs, and –keepuser options to limit what files will be affected.

To see what files kscrubber will modify on your system without making any actual changes, include the –sim (simulate) option:

sudo kscrubber --clean --sim

The –backup option may be used to create backup copies of all files that are changed. Be sure the backup folder you specify has ample space – if you have a large amount of temporary files which kscrubber deletes, the backup can be quite large. If for any reason a backup of a file fails, kscrubber will halt. Using the backup option is generally not necessary and is provided for testing purposes. Be advised that by using the backup option, you will be creating additional copies of the private data you are seeking to remove, so be sure to destroy the backup folder when finished. Also note that if the –backup option is used with the –sim option, backup files WILL be created. (This allows you to create backup copies without doing any actual cleaning. Only those files which would have been removed, edited, or vacuumed are backed up.)

Using the –check option, you can have kscrubber scan your files after cleaning to test for residue. You can also use the –check option alone without –clean, in which case kscrubber will simply run the check without doing any cleaning. You can specify multiple –check options to search for additional strings. kscrubber looks inside of files, as well as examining file and folder names in the specified path. If the search string is found, the files will be listed. No output means nothing was found. For example, to examine /home/myuser for “mytext”:

sudo kscrubber --check "/home/myuser" "mytext"

Keep in mind that the kscrubber check may drift to other filesystems if you have links present, and/or encounter very large files which may slow it down. To have better control over such issues, consider using the find and grep commands directly. Use ‘man grep’ and ‘man find’ for help.

Details

This section describes in more detail what kscrubber will do and not do. Also note that you can open the script in your text editor and examine it.

Necessarily, our computer systems are loaded with private information. kscrubber can’t remove all of it, because you probably intend some of it to remain on your system. What kscrubber tries to remove is data that tracks your activity and records a history, which you may not know about, while leaving data that you probably do know about and want.

Secure Delete
kscrubber uses the srm program (in two-pass mode by default) to wipe all files it removes with random data. While this is not perfect security on modern filesystems, it helps. For even greater reliability, consider temporarily converting your ext3 filesystem to ext2, as described here. For more information on Secure Delete and what else it can do (such as wipe memory and swap areas), use ‘man srm’, ‘man sfill’, ‘man sswap’, and ‘man smem’.

System Files
By default, kscrubber will remove system logs and some leftover temporary files in /tmp, /var/tmp, and /var/log. In general, it knows which files are safe to remove. Sometimes this can free up substantial disk space. The only reason you might want to retain the system logs is if you are diagnosing a problem on your system and are consulting them. Otherwise, they are safe to delete and your system will create new ones as new activity or problems arise. Occassionally deleting a temporary file will cause a program or the system to malfunction. In this case, rebooting will generally solve it. Note that kscrubber does not simply remove ALL temporary files. You may want to examine these folders after cleaning to see what remains. If you use software which generates and leaves temporary files, you may want to add a cleaning command to the kscrubber script to handle these. Or, let me know and I may add it to kscrubber.

Recent Activity
If you open the files ‘~/.kde/share/config/kdeglobals’ and ‘~/.kde/share/config/plasma-desktop-appletsrc’ you might be surprised to see files you accessed or deleted months ago still listed there. In addition, applications such as Kate, Krusader, Gwenview, Ark, Ocular, Kmail, and others add recently used files and folders to their configuration files. Usually the only way to remove this information is to tediously open each file in a text editor and remove it, while being careful to leave program configuration settings intact. kscrubber handles all of this tedious and repetitive editing very quickly.

Firefox
Even when you delete your browsing history, web form history, recently visited links, and other information within Firefox, much of this information may remain in the sqlite database files, marked as invalid data, but still readable with a simple text editor (eg Kate). This is because some builds of Firefox fail to vacuum the sqlite file. To truly delete this information, clear your entire history in Firefox, close Firefox, and run kscrubber. kscrubber uses sqlite3 to vacuum all the non-locked sqlite databases in your system’s home folders. Vacuuming only removes (defrags) invalid data in the files while keeping the valid data. In addition, kscrubber will delete Firefox’s cookies, downloads, and formhistory database files just be to be sure.

Adobe Flash
The security problems of Adobe Flash go on and on, almost as if it is designed to make your system non-secure (maybe it is?) Yet one of the most blatant problems is often not known by users. While Firefox and other browsers will offer to delete cookies, they don’t delete Adobe Flash ‘cookies’ (aka “Local Shared Objects”). kscrubber will delete them.

Note: While most users don’t need them and they are routinely abused, LSO’s do have legitimate uses in Flash. Adobe provides a tool for managing and disabling them. While this tool is highly questionable from a security perspective (as most of Flash seems to be), since you must visit their website to modify local files on your computer, it is another option for managing them. See Adobe Flash Player Security and Privacy. There is also a Firefox add-on called “BetterPrivacy” which manages them (no endorsement of these tools is implied).

Other Programs
Here is a list of programs whose system and user files may be affected by kscrubber:
akonadi amule ark arora avidemux epdfview filelight firefox flash gftp googleearth gqview gwenview java k3b kaffeine kate kde kgrab klipper kmail konqueror kpdf krename krunner krusader ktorrent mlocate mplayer nepomuk okular parcellite plasma smplayer soffice soprano speedcrunch vlc

With some programs, kscrubber will only delete some log files to save disk space, while with others it will do more intensive cleaning. Never assume kscrubber has removed all the information you want removed – open relevant files with your text editor, or use kscrubber’s –check option to search the files. If you need more cleaning done, consider adding your own commands to the script.

Also, kscrubber does tests on a few programs, such as Pidgin, to alert you to possible security or privacy problems, but it doesn’t modify the files of these programs.

To find out exactly what kscrubber does with a particular program, open the kscrubber script with your text editor and search for all occurrences of that program’s name. Or, run kscrubber in simulation mode.


Installation Instructions


Follow the standard Script Installation Instructions. Alternatively, for Debian and Ubuntu a deb package and a PPA repository are available.

Before running kscrubber, install required packages using your package manager. For example, on Debian/Ubuntu:

apt-get install secure-delete sqlite3

Arch Linux Note: The secure-delete package is not currently supported by Arch, although it is now available in the AUR. You can also compile it from the source, or you can use deb2targz (in community) to extract the files from a .deb package appropriate for your architecture (just copy the srm, sfill, smem, and sswap files to /usr/bin, and place the *.gz man files in /usr/share/man/man1). You will also need the sqlite3 package from core.

Feedback

How is kscrubber working for you? If there is something else you think kscrubber should clean, please let me know. And if it causes any problems on your system, please let me know that too. Please be as specific as possible.

Related forum posts:

12 Comments »

  1. Thanks for this.

    It has good ‘press’ on the Kubuntu forum and probably others too.

    Neat stuff I just need to read everything up now and try a little customizing.

    Thanks again.

    Comment by spidpw | January 25, 2010 | Reply

  2. Used your latest version of kscrubber today. The update you made so that bookmarked items icons aren’t scrubbed works. Followed the instructions, and when finished, logged in via KDM. Started Konqueror and checked my bookmarks. The icons were still there. Nice job.

    Snowhog
    KFN Global Moderator

    Comment by Paul | February 7, 2010 | Reply

    • Glad it worked better for you – thanks for letting me know.

      Comment by igurublog | February 8, 2010 | Reply

  3. Wow! Nice. Will definitely look at in more detail.
    Firefox has an addon called Better Privacy to handle LSO files. Maybe html5 will take care of the issue ;-) OpenSUSE has sysconfig variables that can be set to clean /tmp and /var/tmp at boot. They also have a nice log rotate procedure. I don’t think many people know they need to run an external utility to manage and clean up their FF data. There certainly is a lot of data littering going on. I’ve seen articles on bleachbit for cleaning. I use Ubuntu and OpenSUSE. I’ll have to see what the package for “secure-delete” is on OpenSUSE.

    Comment by dj | May 14, 2010 | Reply

    • This rather long thread contains my review of Bleachbit.

      http://kubuntuforums.net/forums/index.php?topic=3108110.0

      In short, while it addresses more apps than the default version of kscrubber, it leaves a lot behind because it does not (cannot?) edit config files.

      If you find secure-delete on SUSE I’d be interested to know the details. kscrubber requires srm due to its ability to recurse, unlike shred. There are other similar utilities you could configure the script to use as long as they support recursive removal. Thanks for your feedback.

      Comment by igurublog | May 14, 2010 | Reply

  4. hello IgnorantGuru,

    i’ve been using your script with kubuntu for a bit and i think it is fantastic. the reason for this comment is to let you know that i’ve modified it to work with Sabayon (using KDE still) and i “ported” it to XFCE to use it with Fedora. the one i use with Sabayon is still kscrubber (only change is the secure-delete package, sabayon has srm) but the “port” to XFCE was heavily edited and altered and i changed the name (to xscrubber [lame i know]) i’m using it for myself but i wanted to know if what i did (reference kscrubber and link to this blog in the source and help message) is ok or if i should do something else to give you due credit, in case i share it with someone.

    thanks for your time and for the great script.

    Comment by GabrielYYZ | April 2, 2011 | Reply

    • Great – thanks for sharing. If you make any changes to the script and share it, please change its name – either give it a new one or just add -something onto it (eg kscrubber-gab). Just prevents confusion and collisions. I had plans for an ‘xscrubber’, so you beat me to it. ;)

      Feel free to post a link here to your modified versions – I’m sure others will appreciate them.

      Comment by IgnorantGuru | April 2, 2011 | Reply

  5. thank you very much! This script is great.

    Comment by David | December 14, 2011 | Reply

  6. By the way on Fedora 16, I get the warning message that srm is not installed, but it is installed (v1.2.11) and in the path.

    I guess the test expect an error message when deleting a non-existing file? This is not the case on Fedora.

    I deleted the check and everything works fine.

    Comment by David | December 14, 2011 | Reply

    • On some distros, the ‘srm’ package installs an inferior program which is also called ‘srm’. This could be why kscrubber is reporting it is not installed. On the real srm, source available from here, version should report something like:

          $ srm -V
          srm v3.1 (c) 1997-2003 by van Hauser / THC <vh@thc.org>

      If that’s what you’re using and kscrubber still gives a message, please let me know the output of ‘srm -V’ on your system – thanks.

      Comment by IgnorantGuru | December 15, 2011 | Reply

  7. The following folder in Fedora might be also a candidate for scrubbing. For example it stores every time the name of the pdf document and last page etc in that folder.

    .local/share/gvfs-metadata

    Comment by David | December 14, 2011 | Reply

    • kscrubber is not currently being developed, mainly because I’m no longer using KDE. So it may be out of date with the newer KDE and other software it handles. As some point I’d like to write a more generalized version.

      kscrubber can still be used, even on non-KDE systems, but you should check to make sure it’s cleaning what you want. And it’s easy to add your own cleaning commands to the script – just follow the examples. That was the intention all along – this script can be customized to create your own cleaning script.

      I’ve made a note of the gvfs folder and the next time I work on this I’ll take a look – thanks.

      Comment by IgnorantGuru | December 15, 2011 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 146 other followers