Nepomuk not indexing a large home

I have a large home dir, because I use it for work and personal development, and there are hundreds of thousands of files.
I was wondering why Dolphin’s timeline was not showing the accessed files. I began to look around, and I found a way to get nepomukserver’s log.
First, enable debug output with kdebugdialog. Run it and enable debug for Nepomuk.
Then, restart nepomukserver from a terminal:

qdbus org.kde.NepomukServer /nepomukserver quit
nepomukserver

Then, among all the output, I found a message that gave me a clue:

User limit reached. Please raise the inotify user watch limit

With this, I searched around and found how to increase that limit, in sysctl.conf

sudo sysctl fs.inotify.max_user_watches=<some random high number>

And for a more permanent solution, add that key to /etc/sysctl.conf

fs.inotify.max_user_watches=500000

I picked 500000 because I have lots of files, and also 8GB of RAM, so memory shouldn’t be an issue. Default is 8k, packages for indexing servers usually increase it to 100k.
After this, I restarted Nepomuk, and files started showing in my Dolphin’s timeline. Yay!
For a final wrap-up, I disabled debug and restarted nepomukserver.

Disclosure: I’m not a Nepomuk expert, I’m just showing something that worked for me.

About asoliverez

Free Software developer, KDE e.V. member, political activist at SoftwareLibreConCFK, Zen Buddhist, son of a retired physicist and politician ,and a Literature professor; husband of a beautiful QA consultant, and father of the cutest daughter

6 thoughts on “Nepomuk not indexing a large home

  1. Jeff LaLone
    06/10/2012 at 00:18

    I’m glad I saw this! I had the same problem a while back – each login nepomuk would grind away re-indexing things because, at the end, it hadn’t indexed everything. I couldn’t find a “quick fix” in any of the settings dialogs, so I ended up just disabling the search. Now I’ll turn it back on!

  2. MJD
    06/10/2012 at 02:18

    Be careful when up this value though. I did something similar on my desktop, and certain packages (*cough* mediatomb *cough*) allocate memory based upon how many watches a user can open. Nepomuk does the responsible thing and works perfectly, but the other package consumed something like 4-8G or more.

  3. YoYo
    06/10/2012 at 09:01

    I had to restrict nepomuk indexing to a couple of small dirs (downloads,documents I created…) because after it indexed everything in my home, it ended up with a database a bit over 1GB which is a little too much on my laptop, where I constantly struggle with empty space on the drive ;(

    1. 06/10/2012 at 09:10

      You are right, it consumes some space. I didn’t mention that because I have a large enough disk, but it’s a possible problem in small HDs.

  4. moltonel
    08/10/2012 at 08:30

    I already had max_user_watches=524288 (2^19), but after a check I have less than 400k files (and only half of that indexed). Thanks for the tip, but I’ll have to keep searching why file indexing is still not working quite right for me. One thing I’d like to try is a dump+restore of the virtuoso db (2.5Gb on disk, 43Mb in backup form), but it looks dauntingly complicated.

    1. 08/10/2012 at 14:53

      If you don’t have custom tags on it, you could try removing the db completely.
      Also, try the instructions for debugging, and see if the debug output is of any good. That’s how I found this issue in the first place.