chris
Dark Lord of the Sith
Joined: 10 May 2003
Posts: 6262
Location: Outer Space
|
 Posted: Sun Aug 05, 2007 7:47 pm Post subject: How to increase the max number of open files for Linux users |
|
 |
|
| Quote: |  As we all know, Linux sucks. It especially sucks with regards to user limits. |
(from Fun with user limits)
I can fully understand the rage of the person who wrote this - it's sad, but I do can...
I thought the first and last time I would ever have to deal with the problem of "how many processes or files should a user be allowed to open at maximum" would be during kernel compilation. Having the habit of always going through all kernel configuration options (takes me 4-5 hours each time I do it... ), I inevitably stumbled upon such parameters in the good ol' days.
In recent years, I have forgotten the subject completely. This is because now, in kernels 2.4, 2.6 and on, these limits are handled by Linux automatically.
Which is a Good Thing - if there were not those cases were those forgotten details would come back to bite you in a moment you actually cannot afford the luxury of delving deeply into, say, kernel internals...
So one day you get the error message:
| Code: |
pi-gtk-3236.so: cannot open shared object file: Too many open files
|
and you must face it: either this is a "false positive" (an alarm about some event that actually has NOT happened), or it is indeed true and you have too many open files - but how do you increase that limit then?
Running your own Linux system means you want to take full control of your system, right? But with full control comes full responsibility and full configurability. Face it and learn the gory details:
For 2.4 and 2.6 kernels, per process and total system file limits are defined in include/linux/limits.h and include/linux/fs.h. To change these limits, go to the include/linux directory inside the code tree of your kernel, change these constants and recompile your kernel:
| Code: |
NR_OPEN = maximum number of open files per process
NR_FILE = total number of files that can be open in the system at any time
|
Before you do it, wait a minute! Do you know if this will bring you more than you already have? How do you check before you plunge into this adventure?
Here's how:
| Code: |
cat /proc/sys/fs/file-max
288217
|
The answer (from my system) of "288217" is plenty, isn't it? Let's look at the code in includes/linux/fs.h a bit:
| Code: |
#define NR_OPEN (1024*1024) /* Absolute upper limit on fd num */
#define INR_OPEN 1024 /* Initial setting for nfile rlimits */
|
So I may not need to touch NR_OPEN at all, but if I want that the default value for the maximum of open file descriptors is more than 1024, without having to set it programmatically in .bashrc, in a system profile or whatever, then I do have to increase INR_OPEN above and recompile the kernel!
However, since recompiling the kernel is connected to many other tasks on my system (recompile the Nvidia graphics driver, recompile ALSA sound modules, recompile VMware module...), I am reluctant to follow this direction.
So what is the alternative?
The alternative is, as already hinted, to set those limits each time you boot, for each user. Here's how:
If you are user X, you can see the current limit for you with:
If you have not used the ulimit command in your user profile, .bashrc file or anything to tweak the maximum number of open file descriptors, then you should see the value of INR_OPEN as set in the includes/linux/fs.h file, that is: 1024.
In today's world with so many open (and heavy) programs around on any given desktop (and especially on the desktop of a software developer...), this number is bound to be an underestimation of what you actually need.
Let's check it! How many open files do you have? This is the realm of the "lsof" command:
will show you all files that are open by user chris. I will not show you the outcome, because they are indeed plenty! 
However, I can pipe the output of lsof to the wc (word count) command, to get the number of open files by user chris, as follows:
| Code: |
lsof -u chris | wc -l
6652
|
WOW! 6652 open files at the time of writing this! And I am not actually doing anything really special at this time!
So can you increase the number of open file descriptors for a user?
Let me put a note here, before we proceed: why do I talk about "file descriptors" and not simply "files"? This is because there is quite a difference between the number of open files and the number of file descriptors per process:
| Quote: | A file descriptor is a data structure used by a program to get a handle on a file, the most well known being 0,1,2 for standard in, standard out, and standard error.
...
'lsof' gives the number of open files. An open file may be a regular file, a directory, a block special file, a character special file, an executing text reference, a library, a stream or a network file. Though a file is open, it might not have a file descriptor associated with it - such as current working directories, memory mapped files and executable text files. 'lsof | wc -l' gives the current number of open files.. Therefore, there will be a difference in the number of current open files and the number of current file descriptors/handles.
|
(from What is an open file?)
So I may have 6000 files open, but not need as many file descriptors. Still, I don't feel comfortable, especially when I encounter errors like the "Too many open files" above, so I want to increase my own max limit.
To increase the maximum number of file descriptors allowed for a user, just put
in his profile file, or .bashrc. Now, you have to be careful here: if you are the only user on your system (not counting users needed by some programs, like your servlet container (tomcat) or web server (wwwrun) etc.) and if you want to first make 200% sure that you have enough file descriptors, then go with such a large number as 32768 above. But setting a limit that high for all users in your system, may lead to system degradation.
As you guess, there is a compromise you have to strike here - and I am sure you will, at the end of all ends... 
_________________
Regards
Chris Karakas
www.karakas-online.de
Last edited by chris on Sun Aug 05, 2007 10:26 pm; edited 1 time in total |
|
chris
Dark Lord of the Sith
Joined: 10 May 2003
Posts: 6262
Location: Outer Space
|
| Posted: Sun Aug 05, 2007 8:44 pm Post subject: |
|
|
|
Wait a minute! Does this mean that by just putting
in my profile file, or .bashrc, I can increase my limit as I please? What if I put
there? What will happen then? Will I really get that many file descriptors if I need them?
Not exactly. This is where the /etc/security/limits.conf comes into play! It contains the absolute maximums a user can set for himself with the ulimit command. For the file descriptors, for example, I have set the "nofile" option as follows in my /etc/security/limits.conf:
| Code: |
* soft nofile 1024
* hard nofile 2048
root soft nofile 2048
root hard nofile 32768
chris soft nofile 1024
chris hard nofile 32768
tomcat soft nofile 2048
tomcat hard nofile 8192
|
User chris will start with a maximum of 1024 (that is the "soft" line), but if he likes, he can set it up to 32768 with a ulimit command (in his profile, .bashrc file, on the command line and so on).
User tomcat will start with a maximum of 2048 (just to be sure all those webapps get what they need) and can even increase it with a ulimit command up to a hard limit of 8192.
root starts with 2048. An
| Code: |
lsof -u root | wc -l
|
gives me a number of 1947 open files - not file descriptors, sure, but for the sake of example let's say root needs that much. The limit can be increased up to a hard limit of 32768.
All other users (the * lines in /etc/security/limits.conf above) start with a max of 1024 and may increase it up to 2048.
But again...suppose I am root and can edit the /etc/security/limits.conf file. Does that mean I can enter as high limits as I please there?
Again no - of course not. Your "limit of limits" in this case is the value you got when you did
| Code: |
cat /proc/sys/fs/file-max
|
at the start of this discussion. This is what your kernel uses. To change the kernel parameter on the fly, you can "echo" the number you wish to /proc/sys/fs/file-max as follows (do it with caution please):
| Code: |
echo "300000" > /proc/sys/fs/file-max
|
Now test again:
| Code: |
cat /proc/sys/fs/file-max
300000
|
and you get the new limit you set.
To know how many file descriptors are being used, do a
| Code: |
cat /proc/sys/fs/file-nr
|
You get an output like this:
| Code: |
8667 3145 288217
| | |__ maximum number of file descriptors allowed on the system
| |
| |__ total free allocated file descriptors
|
|__ total allocated file descriptors
|
Equipped with all that know-how, you are now in a position to quickly set new file descriptor limits for yourself and the users of your system.
 PS.: So how about that error
| Code: |
pi-gtk-3236.so: cannot open shared object file: Too many open files
|
that sent me on this long journey in search of new limits?
Well, it turned out that, even with a
in my .bashrc, the error persisted. Now this is certainly something deeper than just a problem of maximum number of open files. So I have come all this way only to find out that this does not solve my problem. At least now I know that is is NOT the number of open file descriptors that causes it.
What is it then?
Well, dear reader, this is a story for another time, another day... (see _X11TransSocketOpen: socket() failed for local).
_________________
Regards
Chris Karakas
www.karakas-online.de |
|
FAQ
Forum Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
