Too many open files error

Taking care of your Linux box.

Too many open files error

Postby kashif » Wed Feb 18, 2009 12:22 pm

assalam o alaikum,

I have a query and will be thankful for some good reply,


On asterisk server (centos 5.0) I am finding the frequent message in asterisk logs


Feb 17 11:38:38 NOTICE[3697] manager.c: Accept returned -1: Too many open files


when I check for open files

# lsof |wc -l
4022


I have increased the Open file limit for kernel much higher

# sysctl -a |grep file-max
fs.file-max = 655910



But still I am facing the" Too many open files" error

What may be the reason for that or what other parameter is needed to tune for avoidng the error.
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara

Postby osama » Wed Feb 18, 2009 2:48 pm

Check here. This might help

http://tldp.org/LDP/solrhe/Securing-Opt ... x4733.html

How did u set the open files limit and what distro u r using?

I once tried to increase open files limit according to my requirements using PAM but was unseccessful. And I concluded in Vanilla distros PAM does not include this support and you have to compile the kernel. I was working for Squid and so the other solution was to compile the squid with increased open files support.

If u find a solution tell us too.

Otherwise u have to manage sessions properly in asterisk.
osama
Havaldaar
 
Posts: 117
Joined: Fri Aug 22, 2008 9:08 am

Postby kashif » Wed Feb 18, 2009 4:17 pm

Distro is Centos 5.0 as mentioned earlier.

and to increase the system wide limit

vi /etc/sysctl.conf

fs.file-max = 655910


and #sysctl -p will reread this file.
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara

Postby osama » Wed Feb 18, 2009 5:44 pm

google and try "ulimit -a". Learn what actually it can do.

Start Asterisk from an init script. Add this in script

"ulimit -n 64000"

prior to starting Asterisk.
osama
Havaldaar
 
Posts: 117
Joined: Fri Aug 22, 2008 9:08 am

Re:

Postby LinuxFreaK » Wed Feb 18, 2009 5:51 pm

Dear
Salam,

Post output of following command.

# ls -l /proc/`cat /var/run/asterisk.pid`/fd/ | wc -l

Further more take a look into /etc/security/limits.conf

You can use ulimit command and restart your asterisk.

# ulimit -n 4096

Best Regards.
Farrukh Ahmed
LinuxFreaK
Site Admin
 
Posts: 5132
Joined: Fri May 02, 2003 10:24 am
ICQ: 82075802
Website: http://www.linuxpakistan.net/wiki/index.php?pagename=LinuxFreak
WLM: f4fahmed@hotmail.com
Yahoo Messenger: f4fahmed@yahoo.com
AOL: linuxpakistan@aol.com
Location: Karachi

Postby kashif » Thu Feb 19, 2009 11:59 am

Thanks Furrkuh for you reply,

below is the output

]# cat /var/run/asterisk.pid
2630

[root@vicidial ~]# ls -l /proc/2630/fd/ | wc -l
1017


and


ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
max nice (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 32758
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
max rt priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 32758
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara

Postby osama » Thu Feb 19, 2009 12:17 pm

kashif wrote:open files (-n) 1024


You didnt try ulimit -n stuff ? This will increase open files limit and solve your problem
osama
Havaldaar
 
Posts: 117
Joined: Fri Aug 22, 2008 9:08 am

Postby kashif » Thu Feb 19, 2009 1:36 pm

aoa,

Now doing that and will reply soon.

w/s
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara

Postby kashif » Thu Feb 19, 2009 3:41 pm

aoa,

Problem still persist.

I have increased the limit and even rebooted the machine.

# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
max nice (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 32758
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
max rt priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 32758
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited




Entered the following enty
* soft nofile 4096
* hard nofile 4096

in

/etc/security/limits.conf


Now pictures is as below

Aserisk is running by root user


PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COM
2620 root 15 0 106m 37m 4564 S 32 1.9 12:53.70 asterisk


#ls -l /proc/2620/fd/ |wc -l
1023


#lsof -u root |wc -l
4001



Still facing error

Feb 19 15:16:46 NOTICE[2624] manager.c: Accept returned -1: Too many open files
Feb 19 15:16:46 NOTICE[2624] manager.c: Accept returned -1: Too many open files
Feb 19 15:16:46 NOTICE[2624] manager.c: Accept returned -1: Too many open files



One thing which I am suspecting is that I have set the file open limit size to 4096.

and previous command #lsof -u root |wc -l
4000


indicating that 4000 files are opened by root user.



What maximum limit can I set for ulimit -n ?, mean what parameter memory , files system inodes etc need to be consider before setting its value too HIGH.
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara

Postby osama » Thu Feb 19, 2009 5:13 pm

This might help

http://threebit.net/mail-archive/asteri ... 10747.html

I am usging this

# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
pending signals (-i) 1024
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
osama
Havaldaar
 
Posts: 117
Joined: Fri Aug 22, 2008 9:08 am

Postby kashif » Sat Feb 21, 2009 9:34 pm

Thanks osama :lol:

It solved a big problem.

Above is very good link to the related problem.

The solution was to add the command "ulimit -n 65536" to the init script, prior to starting Asterisk.
**********************************************

As-Salaatu was-Salaamu Alaika Ya Sayyidi Ya Rasool ALLAH

**********************************************
kashif
Naib Subedar
 
Posts: 305
Joined: Wed Oct 15, 2003 2:44 am
Location: Okara


Return to “%s” Administration

Who is online

Users browsing this forum: No registered users and 2 guests

cron