Apache's “OPTIONS * HTTP/1.0” running 100% CPU - Runaway httpd processIs Apache stalled? /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)”What can cause Apache HTTPD to use 100% CPU indefinitelyemond process running at 100%How to avoid Apache “options * http/1.0”Hung Apache Processes after POSTHigh CPU from httpd processApache HTTPD process cumulative cpu timeOPTIONS * HTTP/1.0 in access_logApache 2.2 eventually using all memory (worker mpm)Apache status says stopped but still httpd process runninghttpd processes cpu spike to 100%+ every xx seconds

Convert Numbers To Emoji Math

Subnumcases as a part of align

Why increasing of the temperature of the objects like wood, paper etc. doesn't fire them?

Was there a dinosaur-counter in the original Jurassic Park movie?

Primes in a Diamond

How long did it take Captain Marvel to travel to Earth?

Endgame puzzle: How to avoid stalemate and win?

Is there precedent or are there procedures for a US president refusing to concede to an electoral defeat?

Can I combine SELECT TOP() with the IN operator?

Guess the number game (Python)

Which "exotic salt" can lower water's freezing point by –70 °C?

Is it normal for gliders not to have attitude indicators?

In "Avengers: Endgame", what does this name refer to?

Picking a theme as a discovery writer

What is more safe for browsing the web: PC or smartphone?

Is there a reason why Turkey took the Balkan territories of the Ottoman Empire, instead of Greece or another of the Balkan states?

How important are good looking people in a novel/story?

Why does blending blueberries, milk, banana and vanilla extract cause the mixture to have a yogurty consistency?

All of my Firefox add-ons been disabled suddenly, how can I re-enable them?

Why are condenser mics so much more expensive than dynamics?

Copper as an adjective to refer to something made of copper

Has the United States ever had a non-Christian President?

While drilling into kitchen wall, hit a wire - any advice?

Emergency stop in plain TeX, pdfTeX, XeTeX and LuaTeX?



Apache's “OPTIONS * HTTP/1.0” running 100% CPU - Runaway httpd process


Is Apache stalled? /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)”What can cause Apache HTTPD to use 100% CPU indefinitelyemond process running at 100%How to avoid Apache “options * http/1.0”Hung Apache Processes after POSTHigh CPU from httpd processApache HTTPD process cumulative cpu timeOPTIONS * HTTP/1.0 in access_logApache 2.2 eventually using all memory (worker mpm)Apache status says stopped but still httpd process runninghttpd processes cpu spike to 100%+ every xx seconds






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








3















I have this recurring issue where an httpd processes will randomly start running at 100% CPU. Often other httpd processes will join in, and it will continue until I restart Apache. Oddly, the thing it's running 100% CPU on is "OPTIONS * HTTP/1.0". Here's the output from one of these:



9-0 38787 1/9/391 C 103.14 1323 7 0.0 0.08 4.11 ::1 www.mysite.com OPTIONS * HTTP/1.0


CPU is at 103.14% and it's been 1323 seconds since its last request. It's also stuck in 'C' - closing connection state.



Here's another case where other processes join in running 100% CPU:



0-0 12792 0/33/64 W 95.73 1097 0 0.0 0.10 0.39 66.68.237.216 www.mysite.com POST /page_a.php HTTP/1.1
9-0 12795 1/6/15 C 94.42 1174 0 0.0 0.03 0.07 ::1 www.myserver.com OPTIONS * HTTP/1.0
19-0 12986 0/4/41 W 95.67 1011 0 0.0 0.03 0.24 81.237.216.111 www.mysite.com POST /page_b.php HTTP/1.1
20-0 12720 0/10/10 W 94.32 1220 0 0.0 0.03 0.03 187.184.103.218 www.mysite.com POST /page_a.php HTTP/1.1


My setup is that this is on OS X Lion 10.4.7. I'm running Apache2 with PHP 5.3. Server Version: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.10 with Suhosin-Patch



Some potentially relevant http.conf settings:



MaxRequestsPerChild 100000
Timeout 300
KeepAlive On
KeepAliveTimeout 8
MaxKeepAliveRequests 100


When I sample the runaway processes, I see each of them show _spin_lock and kevent like so:



 Sort by top of stack, same collapsed (when >= 5):
_spin_lock$VARIANT$mp (in libsystem_c.dylib) 4358
kevent (in libsystem_kernel.dylib) 2179


Here is some of the non-repeating Apple code I see:



Call graph:
2413 Thread_1080146 DispatchQueue_1: com.apple.main-thread (serial)
+ 2413 start (in httpd) + 52 [0x10f364794]
+ 2413 main (in httpd) + 4045 [0x10f37048d]
+ 2413 ap_mpm_run (in httpd) + 1740 [0x10f3aaabc]
+ 2413 perform_idle_server_maintenance (in httpd) + 703 [0x10f3aa38f]
+ 2413 make_child (in httpd) + 435 [0x10f3aa003]
+ 2413 child_main (in httpd) + 1831 [0x10f3a9e47]
+ 2413 clean_child_exit (in httpd) + 49 [0x10f3a8d31]
+ 2413 apr_pool_destroy (in libapr-1.0.dylib) + 52 [0x10f45761b]
+ 2413 ??? (in libapr-1.0.dylib) load address 0x10f44b000 + 0xb7b4 [0x10f4567b4]
+ 2413 php_apache_child_shutdown (in libphp5.so) + 17 [0x10fa3842f]
+ 2413 php_module_shutdown_wrapper (in libphp5.so) + 9 [0x10f97021d]
+ 2413 php_module_shutdown (in libphp5.so) + 35 [0x10f970167]
+ 2413 zend_shutdown (in libphp5.so) + 57 [0x10f9c0692]
+ 2413 zend_hash_destroy (in libphp5.so) + 53 [0x10f9cb19a]
+ 2413 destroy_op_array (in libphp5.so) + 271 [0x10f9b91c0]
+ 2413 _efree (in libphp5.so) + 52 [0x10f9a6312]
+ 2413 _zend_mm_free_canary_int (in libphp5.so) + 473 [0x10f9db899]
+ 2413 free (in libsystem_c.dylib) + 71 [0x7fff8f86170e]
+ 2413 szone_size_try_large (in libsystem_c.dylib) + 37 [0x7fff8f8240f9]
+ 2413 _spin_lock$VARIANT$mp (in libsystem_c.dylib) + 30,25,... [0x7fff8f86336e,0x7fff8f863369,...]
2413 Thread_1080153 DispatchQueue_2: com.apple.libdispatch-manager (serial)
2413 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff908fc31a]
2413 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 [0x7fff908fd78a]
2413 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8ed047e6]


And then stuff like this:



 0x10f4ce000 - 0x10f4d0ff7 mod_reqtimeout.so (??? - ???) <035F872B-8196-3CCE-A4D0-AA8D5C1550EC> /usr/libexec/apache2/mod_reqtimeout.so
0x10f4d4000 - 0x10f4d8ff7 mod_ext_filter.so (??? - ???) <F33A3409-BFBA-3C8F-9FC9-C1BBC17DBE3F> /usr/libexec/apache2/mod_ext_filter.so
0x10f4dd000 - 0x10f4eaff7 mod_include.so (??? - ???) <70E541B9-A864-3FE1-AB85-EBF632FFD376> /usr/libexec/apache2/mod_include.so
0x10f4ef000 - 0x10f4f2ff7 mod_filter.so (??? - ???) <2093EE45-E335-3B36-A6BA-6EA4EB7E483C> /usr/libexec/apache2/mod_filter.so
0x10f4f6000 - 0x10f4f8ff7 mod_substitute.so (??? - ???) <9ED1AB37-EE13-39DC-AB97-98A2B39555B0> /usr/libexec/apache2/mod_substitute.so
0x10f4fc000 - 0x10f501ff7 mod_deflate.so (??? - ???) <E334FEA5-BB4E-36BE-A364-DCE248793345> /usr/libexec/apache2/mod_deflate.so
0x10f506000 - 0x10f50bfff mod_log_config.so (??? - ???) <61EA3051-8D4A-3A00-B7BC-C68E18CA9479> /usr/libexec/apache2/mod_log_config.so
0x10f511000 - 0x10f512fef mod_log_forensic.so (??? - ???) <06654BB4-CA2A-3D70-B759-12191119E5C7> /usr/libexec/apache2/mod_log_forensic.so
0x10f516000 - 0x10f516ff7 mod_logio.so (??? - ???) <A3CE3AED-D5FE-3947-8CEB-0A8F4C534E86> /usr/libexec/apache2/mod_logio.so
0x10f51a000 - 0x10f51aff7 mod_env.so (??? - ???) <DE58A08A-C6C9-307B-A077-03A9F1AA14C5> /usr/libexec/apache2/mod_env.so
0x10f51e000 - 0x10f524fff mod_mime_magic.so (??? - ???) <1737F398-6315-31B4-B8B4-57F590F07268> /usr/libexec/apache2/mod_mime_magic.so
0x10f529000 - 0x10f52aff7 mod_cern_meta.so (??? - ???) <B0D4FF96-C96D-3F3A-AECD-431F22A998FA> /usr/libexec/apache2/mod_cern_meta.so
0x10f52e000 - 0x10f52ffff mod_expires.so (??? - ???) <FE97E6F0-3B84-3CAC-8873-3B1525142041> /usr/libexec/apache2/mod_expires.so
0x10f533000 - 0x10f536ff7 mod_headers.so (??? - ???) <5701D330-D777-3AAF-AEEF-F02D067F851E> /usr/libexec/apache2/mod_headers.so
0x10f53b000 - 0x10f53cfff mod_ident.so (??? - ???) <5FDFBB79-3A0C-3439-BF71-74E6A3A4B7AC> /usr/libexec/apache2/mod_ident.so
0x10f540000 - 0x10f542ff7 mod_usertrack.so (??? - ???) <09F36BB5-4F8D-339B-AA52-5FA23A946837> /usr/libexec/apache2/mod_usertrack.so
0x10f546000 - 0x10f547fff mod_setenvif.so (??? - ???) <D312073B-0320-3810-9535-3AB8EBF83659> /usr/libexec/apache2/mod_setenvif.so
0x10f54b000 - 0x10f54cff7 mod_version.so (??? - ???) <4BF2E21C-E452-340E-A6B4-196DBD731CA8> /usr/libexec/apache2/mod_version.so
0x10f550000 - 0x10f566fff mod_proxy.so (??? - ???) <0169F3B2-A81A-3E23-92FE-8E9B92C38795> /usr/libexec/apache2/mod_proxy.so
0x10f56e000 - 0x10f576ff7 mod_proxy_http.so (??? - ???) <F5B55152-D3D1-34AE-A95A-029F01476A54> /usr/libexec/apache2/mod_proxy_http.so
0x10f57c000 - 0x10f57efff mod_proxy_scgi.so (??? - ???) <DFD46BF1-F69D-383B-8B33-5273DD8037E4> /usr/libexec/apache2/mod_proxy_scgi.so
0x10f583000 - 0x10f589ff7 mod_proxy_balancer.so (??? - ???) <BC9C283D-8852-3016-8142-96A5675956FA> /usr/libexec/apache2/mod_proxy_balancer.so


I can post more details as requested.










share|improve this question
























  • I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

    – Stefan Lasiewski
    May 27 '12 at 18:13

















3















I have this recurring issue where an httpd processes will randomly start running at 100% CPU. Often other httpd processes will join in, and it will continue until I restart Apache. Oddly, the thing it's running 100% CPU on is "OPTIONS * HTTP/1.0". Here's the output from one of these:



9-0 38787 1/9/391 C 103.14 1323 7 0.0 0.08 4.11 ::1 www.mysite.com OPTIONS * HTTP/1.0


CPU is at 103.14% and it's been 1323 seconds since its last request. It's also stuck in 'C' - closing connection state.



Here's another case where other processes join in running 100% CPU:



0-0 12792 0/33/64 W 95.73 1097 0 0.0 0.10 0.39 66.68.237.216 www.mysite.com POST /page_a.php HTTP/1.1
9-0 12795 1/6/15 C 94.42 1174 0 0.0 0.03 0.07 ::1 www.myserver.com OPTIONS * HTTP/1.0
19-0 12986 0/4/41 W 95.67 1011 0 0.0 0.03 0.24 81.237.216.111 www.mysite.com POST /page_b.php HTTP/1.1
20-0 12720 0/10/10 W 94.32 1220 0 0.0 0.03 0.03 187.184.103.218 www.mysite.com POST /page_a.php HTTP/1.1


My setup is that this is on OS X Lion 10.4.7. I'm running Apache2 with PHP 5.3. Server Version: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.10 with Suhosin-Patch



Some potentially relevant http.conf settings:



MaxRequestsPerChild 100000
Timeout 300
KeepAlive On
KeepAliveTimeout 8
MaxKeepAliveRequests 100


When I sample the runaway processes, I see each of them show _spin_lock and kevent like so:



 Sort by top of stack, same collapsed (when >= 5):
_spin_lock$VARIANT$mp (in libsystem_c.dylib) 4358
kevent (in libsystem_kernel.dylib) 2179


Here is some of the non-repeating Apple code I see:



Call graph:
2413 Thread_1080146 DispatchQueue_1: com.apple.main-thread (serial)
+ 2413 start (in httpd) + 52 [0x10f364794]
+ 2413 main (in httpd) + 4045 [0x10f37048d]
+ 2413 ap_mpm_run (in httpd) + 1740 [0x10f3aaabc]
+ 2413 perform_idle_server_maintenance (in httpd) + 703 [0x10f3aa38f]
+ 2413 make_child (in httpd) + 435 [0x10f3aa003]
+ 2413 child_main (in httpd) + 1831 [0x10f3a9e47]
+ 2413 clean_child_exit (in httpd) + 49 [0x10f3a8d31]
+ 2413 apr_pool_destroy (in libapr-1.0.dylib) + 52 [0x10f45761b]
+ 2413 ??? (in libapr-1.0.dylib) load address 0x10f44b000 + 0xb7b4 [0x10f4567b4]
+ 2413 php_apache_child_shutdown (in libphp5.so) + 17 [0x10fa3842f]
+ 2413 php_module_shutdown_wrapper (in libphp5.so) + 9 [0x10f97021d]
+ 2413 php_module_shutdown (in libphp5.so) + 35 [0x10f970167]
+ 2413 zend_shutdown (in libphp5.so) + 57 [0x10f9c0692]
+ 2413 zend_hash_destroy (in libphp5.so) + 53 [0x10f9cb19a]
+ 2413 destroy_op_array (in libphp5.so) + 271 [0x10f9b91c0]
+ 2413 _efree (in libphp5.so) + 52 [0x10f9a6312]
+ 2413 _zend_mm_free_canary_int (in libphp5.so) + 473 [0x10f9db899]
+ 2413 free (in libsystem_c.dylib) + 71 [0x7fff8f86170e]
+ 2413 szone_size_try_large (in libsystem_c.dylib) + 37 [0x7fff8f8240f9]
+ 2413 _spin_lock$VARIANT$mp (in libsystem_c.dylib) + 30,25,... [0x7fff8f86336e,0x7fff8f863369,...]
2413 Thread_1080153 DispatchQueue_2: com.apple.libdispatch-manager (serial)
2413 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff908fc31a]
2413 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 [0x7fff908fd78a]
2413 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8ed047e6]


And then stuff like this:



 0x10f4ce000 - 0x10f4d0ff7 mod_reqtimeout.so (??? - ???) <035F872B-8196-3CCE-A4D0-AA8D5C1550EC> /usr/libexec/apache2/mod_reqtimeout.so
0x10f4d4000 - 0x10f4d8ff7 mod_ext_filter.so (??? - ???) <F33A3409-BFBA-3C8F-9FC9-C1BBC17DBE3F> /usr/libexec/apache2/mod_ext_filter.so
0x10f4dd000 - 0x10f4eaff7 mod_include.so (??? - ???) <70E541B9-A864-3FE1-AB85-EBF632FFD376> /usr/libexec/apache2/mod_include.so
0x10f4ef000 - 0x10f4f2ff7 mod_filter.so (??? - ???) <2093EE45-E335-3B36-A6BA-6EA4EB7E483C> /usr/libexec/apache2/mod_filter.so
0x10f4f6000 - 0x10f4f8ff7 mod_substitute.so (??? - ???) <9ED1AB37-EE13-39DC-AB97-98A2B39555B0> /usr/libexec/apache2/mod_substitute.so
0x10f4fc000 - 0x10f501ff7 mod_deflate.so (??? - ???) <E334FEA5-BB4E-36BE-A364-DCE248793345> /usr/libexec/apache2/mod_deflate.so
0x10f506000 - 0x10f50bfff mod_log_config.so (??? - ???) <61EA3051-8D4A-3A00-B7BC-C68E18CA9479> /usr/libexec/apache2/mod_log_config.so
0x10f511000 - 0x10f512fef mod_log_forensic.so (??? - ???) <06654BB4-CA2A-3D70-B759-12191119E5C7> /usr/libexec/apache2/mod_log_forensic.so
0x10f516000 - 0x10f516ff7 mod_logio.so (??? - ???) <A3CE3AED-D5FE-3947-8CEB-0A8F4C534E86> /usr/libexec/apache2/mod_logio.so
0x10f51a000 - 0x10f51aff7 mod_env.so (??? - ???) <DE58A08A-C6C9-307B-A077-03A9F1AA14C5> /usr/libexec/apache2/mod_env.so
0x10f51e000 - 0x10f524fff mod_mime_magic.so (??? - ???) <1737F398-6315-31B4-B8B4-57F590F07268> /usr/libexec/apache2/mod_mime_magic.so
0x10f529000 - 0x10f52aff7 mod_cern_meta.so (??? - ???) <B0D4FF96-C96D-3F3A-AECD-431F22A998FA> /usr/libexec/apache2/mod_cern_meta.so
0x10f52e000 - 0x10f52ffff mod_expires.so (??? - ???) <FE97E6F0-3B84-3CAC-8873-3B1525142041> /usr/libexec/apache2/mod_expires.so
0x10f533000 - 0x10f536ff7 mod_headers.so (??? - ???) <5701D330-D777-3AAF-AEEF-F02D067F851E> /usr/libexec/apache2/mod_headers.so
0x10f53b000 - 0x10f53cfff mod_ident.so (??? - ???) <5FDFBB79-3A0C-3439-BF71-74E6A3A4B7AC> /usr/libexec/apache2/mod_ident.so
0x10f540000 - 0x10f542ff7 mod_usertrack.so (??? - ???) <09F36BB5-4F8D-339B-AA52-5FA23A946837> /usr/libexec/apache2/mod_usertrack.so
0x10f546000 - 0x10f547fff mod_setenvif.so (??? - ???) <D312073B-0320-3810-9535-3AB8EBF83659> /usr/libexec/apache2/mod_setenvif.so
0x10f54b000 - 0x10f54cff7 mod_version.so (??? - ???) <4BF2E21C-E452-340E-A6B4-196DBD731CA8> /usr/libexec/apache2/mod_version.so
0x10f550000 - 0x10f566fff mod_proxy.so (??? - ???) <0169F3B2-A81A-3E23-92FE-8E9B92C38795> /usr/libexec/apache2/mod_proxy.so
0x10f56e000 - 0x10f576ff7 mod_proxy_http.so (??? - ???) <F5B55152-D3D1-34AE-A95A-029F01476A54> /usr/libexec/apache2/mod_proxy_http.so
0x10f57c000 - 0x10f57efff mod_proxy_scgi.so (??? - ???) <DFD46BF1-F69D-383B-8B33-5273DD8037E4> /usr/libexec/apache2/mod_proxy_scgi.so
0x10f583000 - 0x10f589ff7 mod_proxy_balancer.so (??? - ???) <BC9C283D-8852-3016-8142-96A5675956FA> /usr/libexec/apache2/mod_proxy_balancer.so


I can post more details as requested.










share|improve this question
























  • I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

    – Stefan Lasiewski
    May 27 '12 at 18:13













3












3








3








I have this recurring issue where an httpd processes will randomly start running at 100% CPU. Often other httpd processes will join in, and it will continue until I restart Apache. Oddly, the thing it's running 100% CPU on is "OPTIONS * HTTP/1.0". Here's the output from one of these:



9-0 38787 1/9/391 C 103.14 1323 7 0.0 0.08 4.11 ::1 www.mysite.com OPTIONS * HTTP/1.0


CPU is at 103.14% and it's been 1323 seconds since its last request. It's also stuck in 'C' - closing connection state.



Here's another case where other processes join in running 100% CPU:



0-0 12792 0/33/64 W 95.73 1097 0 0.0 0.10 0.39 66.68.237.216 www.mysite.com POST /page_a.php HTTP/1.1
9-0 12795 1/6/15 C 94.42 1174 0 0.0 0.03 0.07 ::1 www.myserver.com OPTIONS * HTTP/1.0
19-0 12986 0/4/41 W 95.67 1011 0 0.0 0.03 0.24 81.237.216.111 www.mysite.com POST /page_b.php HTTP/1.1
20-0 12720 0/10/10 W 94.32 1220 0 0.0 0.03 0.03 187.184.103.218 www.mysite.com POST /page_a.php HTTP/1.1


My setup is that this is on OS X Lion 10.4.7. I'm running Apache2 with PHP 5.3. Server Version: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.10 with Suhosin-Patch



Some potentially relevant http.conf settings:



MaxRequestsPerChild 100000
Timeout 300
KeepAlive On
KeepAliveTimeout 8
MaxKeepAliveRequests 100


When I sample the runaway processes, I see each of them show _spin_lock and kevent like so:



 Sort by top of stack, same collapsed (when >= 5):
_spin_lock$VARIANT$mp (in libsystem_c.dylib) 4358
kevent (in libsystem_kernel.dylib) 2179


Here is some of the non-repeating Apple code I see:



Call graph:
2413 Thread_1080146 DispatchQueue_1: com.apple.main-thread (serial)
+ 2413 start (in httpd) + 52 [0x10f364794]
+ 2413 main (in httpd) + 4045 [0x10f37048d]
+ 2413 ap_mpm_run (in httpd) + 1740 [0x10f3aaabc]
+ 2413 perform_idle_server_maintenance (in httpd) + 703 [0x10f3aa38f]
+ 2413 make_child (in httpd) + 435 [0x10f3aa003]
+ 2413 child_main (in httpd) + 1831 [0x10f3a9e47]
+ 2413 clean_child_exit (in httpd) + 49 [0x10f3a8d31]
+ 2413 apr_pool_destroy (in libapr-1.0.dylib) + 52 [0x10f45761b]
+ 2413 ??? (in libapr-1.0.dylib) load address 0x10f44b000 + 0xb7b4 [0x10f4567b4]
+ 2413 php_apache_child_shutdown (in libphp5.so) + 17 [0x10fa3842f]
+ 2413 php_module_shutdown_wrapper (in libphp5.so) + 9 [0x10f97021d]
+ 2413 php_module_shutdown (in libphp5.so) + 35 [0x10f970167]
+ 2413 zend_shutdown (in libphp5.so) + 57 [0x10f9c0692]
+ 2413 zend_hash_destroy (in libphp5.so) + 53 [0x10f9cb19a]
+ 2413 destroy_op_array (in libphp5.so) + 271 [0x10f9b91c0]
+ 2413 _efree (in libphp5.so) + 52 [0x10f9a6312]
+ 2413 _zend_mm_free_canary_int (in libphp5.so) + 473 [0x10f9db899]
+ 2413 free (in libsystem_c.dylib) + 71 [0x7fff8f86170e]
+ 2413 szone_size_try_large (in libsystem_c.dylib) + 37 [0x7fff8f8240f9]
+ 2413 _spin_lock$VARIANT$mp (in libsystem_c.dylib) + 30,25,... [0x7fff8f86336e,0x7fff8f863369,...]
2413 Thread_1080153 DispatchQueue_2: com.apple.libdispatch-manager (serial)
2413 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff908fc31a]
2413 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 [0x7fff908fd78a]
2413 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8ed047e6]


And then stuff like this:



 0x10f4ce000 - 0x10f4d0ff7 mod_reqtimeout.so (??? - ???) <035F872B-8196-3CCE-A4D0-AA8D5C1550EC> /usr/libexec/apache2/mod_reqtimeout.so
0x10f4d4000 - 0x10f4d8ff7 mod_ext_filter.so (??? - ???) <F33A3409-BFBA-3C8F-9FC9-C1BBC17DBE3F> /usr/libexec/apache2/mod_ext_filter.so
0x10f4dd000 - 0x10f4eaff7 mod_include.so (??? - ???) <70E541B9-A864-3FE1-AB85-EBF632FFD376> /usr/libexec/apache2/mod_include.so
0x10f4ef000 - 0x10f4f2ff7 mod_filter.so (??? - ???) <2093EE45-E335-3B36-A6BA-6EA4EB7E483C> /usr/libexec/apache2/mod_filter.so
0x10f4f6000 - 0x10f4f8ff7 mod_substitute.so (??? - ???) <9ED1AB37-EE13-39DC-AB97-98A2B39555B0> /usr/libexec/apache2/mod_substitute.so
0x10f4fc000 - 0x10f501ff7 mod_deflate.so (??? - ???) <E334FEA5-BB4E-36BE-A364-DCE248793345> /usr/libexec/apache2/mod_deflate.so
0x10f506000 - 0x10f50bfff mod_log_config.so (??? - ???) <61EA3051-8D4A-3A00-B7BC-C68E18CA9479> /usr/libexec/apache2/mod_log_config.so
0x10f511000 - 0x10f512fef mod_log_forensic.so (??? - ???) <06654BB4-CA2A-3D70-B759-12191119E5C7> /usr/libexec/apache2/mod_log_forensic.so
0x10f516000 - 0x10f516ff7 mod_logio.so (??? - ???) <A3CE3AED-D5FE-3947-8CEB-0A8F4C534E86> /usr/libexec/apache2/mod_logio.so
0x10f51a000 - 0x10f51aff7 mod_env.so (??? - ???) <DE58A08A-C6C9-307B-A077-03A9F1AA14C5> /usr/libexec/apache2/mod_env.so
0x10f51e000 - 0x10f524fff mod_mime_magic.so (??? - ???) <1737F398-6315-31B4-B8B4-57F590F07268> /usr/libexec/apache2/mod_mime_magic.so
0x10f529000 - 0x10f52aff7 mod_cern_meta.so (??? - ???) <B0D4FF96-C96D-3F3A-AECD-431F22A998FA> /usr/libexec/apache2/mod_cern_meta.so
0x10f52e000 - 0x10f52ffff mod_expires.so (??? - ???) <FE97E6F0-3B84-3CAC-8873-3B1525142041> /usr/libexec/apache2/mod_expires.so
0x10f533000 - 0x10f536ff7 mod_headers.so (??? - ???) <5701D330-D777-3AAF-AEEF-F02D067F851E> /usr/libexec/apache2/mod_headers.so
0x10f53b000 - 0x10f53cfff mod_ident.so (??? - ???) <5FDFBB79-3A0C-3439-BF71-74E6A3A4B7AC> /usr/libexec/apache2/mod_ident.so
0x10f540000 - 0x10f542ff7 mod_usertrack.so (??? - ???) <09F36BB5-4F8D-339B-AA52-5FA23A946837> /usr/libexec/apache2/mod_usertrack.so
0x10f546000 - 0x10f547fff mod_setenvif.so (??? - ???) <D312073B-0320-3810-9535-3AB8EBF83659> /usr/libexec/apache2/mod_setenvif.so
0x10f54b000 - 0x10f54cff7 mod_version.so (??? - ???) <4BF2E21C-E452-340E-A6B4-196DBD731CA8> /usr/libexec/apache2/mod_version.so
0x10f550000 - 0x10f566fff mod_proxy.so (??? - ???) <0169F3B2-A81A-3E23-92FE-8E9B92C38795> /usr/libexec/apache2/mod_proxy.so
0x10f56e000 - 0x10f576ff7 mod_proxy_http.so (??? - ???) <F5B55152-D3D1-34AE-A95A-029F01476A54> /usr/libexec/apache2/mod_proxy_http.so
0x10f57c000 - 0x10f57efff mod_proxy_scgi.so (??? - ???) <DFD46BF1-F69D-383B-8B33-5273DD8037E4> /usr/libexec/apache2/mod_proxy_scgi.so
0x10f583000 - 0x10f589ff7 mod_proxy_balancer.so (??? - ???) <BC9C283D-8852-3016-8142-96A5675956FA> /usr/libexec/apache2/mod_proxy_balancer.so


I can post more details as requested.










share|improve this question
















I have this recurring issue where an httpd processes will randomly start running at 100% CPU. Often other httpd processes will join in, and it will continue until I restart Apache. Oddly, the thing it's running 100% CPU on is "OPTIONS * HTTP/1.0". Here's the output from one of these:



9-0 38787 1/9/391 C 103.14 1323 7 0.0 0.08 4.11 ::1 www.mysite.com OPTIONS * HTTP/1.0


CPU is at 103.14% and it's been 1323 seconds since its last request. It's also stuck in 'C' - closing connection state.



Here's another case where other processes join in running 100% CPU:



0-0 12792 0/33/64 W 95.73 1097 0 0.0 0.10 0.39 66.68.237.216 www.mysite.com POST /page_a.php HTTP/1.1
9-0 12795 1/6/15 C 94.42 1174 0 0.0 0.03 0.07 ::1 www.myserver.com OPTIONS * HTTP/1.0
19-0 12986 0/4/41 W 95.67 1011 0 0.0 0.03 0.24 81.237.216.111 www.mysite.com POST /page_b.php HTTP/1.1
20-0 12720 0/10/10 W 94.32 1220 0 0.0 0.03 0.03 187.184.103.218 www.mysite.com POST /page_a.php HTTP/1.1


My setup is that this is on OS X Lion 10.4.7. I'm running Apache2 with PHP 5.3. Server Version: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.10 with Suhosin-Patch



Some potentially relevant http.conf settings:



MaxRequestsPerChild 100000
Timeout 300
KeepAlive On
KeepAliveTimeout 8
MaxKeepAliveRequests 100


When I sample the runaway processes, I see each of them show _spin_lock and kevent like so:



 Sort by top of stack, same collapsed (when >= 5):
_spin_lock$VARIANT$mp (in libsystem_c.dylib) 4358
kevent (in libsystem_kernel.dylib) 2179


Here is some of the non-repeating Apple code I see:



Call graph:
2413 Thread_1080146 DispatchQueue_1: com.apple.main-thread (serial)
+ 2413 start (in httpd) + 52 [0x10f364794]
+ 2413 main (in httpd) + 4045 [0x10f37048d]
+ 2413 ap_mpm_run (in httpd) + 1740 [0x10f3aaabc]
+ 2413 perform_idle_server_maintenance (in httpd) + 703 [0x10f3aa38f]
+ 2413 make_child (in httpd) + 435 [0x10f3aa003]
+ 2413 child_main (in httpd) + 1831 [0x10f3a9e47]
+ 2413 clean_child_exit (in httpd) + 49 [0x10f3a8d31]
+ 2413 apr_pool_destroy (in libapr-1.0.dylib) + 52 [0x10f45761b]
+ 2413 ??? (in libapr-1.0.dylib) load address 0x10f44b000 + 0xb7b4 [0x10f4567b4]
+ 2413 php_apache_child_shutdown (in libphp5.so) + 17 [0x10fa3842f]
+ 2413 php_module_shutdown_wrapper (in libphp5.so) + 9 [0x10f97021d]
+ 2413 php_module_shutdown (in libphp5.so) + 35 [0x10f970167]
+ 2413 zend_shutdown (in libphp5.so) + 57 [0x10f9c0692]
+ 2413 zend_hash_destroy (in libphp5.so) + 53 [0x10f9cb19a]
+ 2413 destroy_op_array (in libphp5.so) + 271 [0x10f9b91c0]
+ 2413 _efree (in libphp5.so) + 52 [0x10f9a6312]
+ 2413 _zend_mm_free_canary_int (in libphp5.so) + 473 [0x10f9db899]
+ 2413 free (in libsystem_c.dylib) + 71 [0x7fff8f86170e]
+ 2413 szone_size_try_large (in libsystem_c.dylib) + 37 [0x7fff8f8240f9]
+ 2413 _spin_lock$VARIANT$mp (in libsystem_c.dylib) + 30,25,... [0x7fff8f86336e,0x7fff8f863369,...]
2413 Thread_1080153 DispatchQueue_2: com.apple.libdispatch-manager (serial)
2413 _dispatch_mgr_thread (in libdispatch.dylib) + 54 [0x7fff908fc31a]
2413 _dispatch_mgr_invoke (in libdispatch.dylib) + 923 [0x7fff908fd78a]
2413 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8ed047e6]


And then stuff like this:



 0x10f4ce000 - 0x10f4d0ff7 mod_reqtimeout.so (??? - ???) <035F872B-8196-3CCE-A4D0-AA8D5C1550EC> /usr/libexec/apache2/mod_reqtimeout.so
0x10f4d4000 - 0x10f4d8ff7 mod_ext_filter.so (??? - ???) <F33A3409-BFBA-3C8F-9FC9-C1BBC17DBE3F> /usr/libexec/apache2/mod_ext_filter.so
0x10f4dd000 - 0x10f4eaff7 mod_include.so (??? - ???) <70E541B9-A864-3FE1-AB85-EBF632FFD376> /usr/libexec/apache2/mod_include.so
0x10f4ef000 - 0x10f4f2ff7 mod_filter.so (??? - ???) <2093EE45-E335-3B36-A6BA-6EA4EB7E483C> /usr/libexec/apache2/mod_filter.so
0x10f4f6000 - 0x10f4f8ff7 mod_substitute.so (??? - ???) <9ED1AB37-EE13-39DC-AB97-98A2B39555B0> /usr/libexec/apache2/mod_substitute.so
0x10f4fc000 - 0x10f501ff7 mod_deflate.so (??? - ???) <E334FEA5-BB4E-36BE-A364-DCE248793345> /usr/libexec/apache2/mod_deflate.so
0x10f506000 - 0x10f50bfff mod_log_config.so (??? - ???) <61EA3051-8D4A-3A00-B7BC-C68E18CA9479> /usr/libexec/apache2/mod_log_config.so
0x10f511000 - 0x10f512fef mod_log_forensic.so (??? - ???) <06654BB4-CA2A-3D70-B759-12191119E5C7> /usr/libexec/apache2/mod_log_forensic.so
0x10f516000 - 0x10f516ff7 mod_logio.so (??? - ???) <A3CE3AED-D5FE-3947-8CEB-0A8F4C534E86> /usr/libexec/apache2/mod_logio.so
0x10f51a000 - 0x10f51aff7 mod_env.so (??? - ???) <DE58A08A-C6C9-307B-A077-03A9F1AA14C5> /usr/libexec/apache2/mod_env.so
0x10f51e000 - 0x10f524fff mod_mime_magic.so (??? - ???) <1737F398-6315-31B4-B8B4-57F590F07268> /usr/libexec/apache2/mod_mime_magic.so
0x10f529000 - 0x10f52aff7 mod_cern_meta.so (??? - ???) <B0D4FF96-C96D-3F3A-AECD-431F22A998FA> /usr/libexec/apache2/mod_cern_meta.so
0x10f52e000 - 0x10f52ffff mod_expires.so (??? - ???) <FE97E6F0-3B84-3CAC-8873-3B1525142041> /usr/libexec/apache2/mod_expires.so
0x10f533000 - 0x10f536ff7 mod_headers.so (??? - ???) <5701D330-D777-3AAF-AEEF-F02D067F851E> /usr/libexec/apache2/mod_headers.so
0x10f53b000 - 0x10f53cfff mod_ident.so (??? - ???) <5FDFBB79-3A0C-3439-BF71-74E6A3A4B7AC> /usr/libexec/apache2/mod_ident.so
0x10f540000 - 0x10f542ff7 mod_usertrack.so (??? - ???) <09F36BB5-4F8D-339B-AA52-5FA23A946837> /usr/libexec/apache2/mod_usertrack.so
0x10f546000 - 0x10f547fff mod_setenvif.so (??? - ???) <D312073B-0320-3810-9535-3AB8EBF83659> /usr/libexec/apache2/mod_setenvif.so
0x10f54b000 - 0x10f54cff7 mod_version.so (??? - ???) <4BF2E21C-E452-340E-A6B4-196DBD731CA8> /usr/libexec/apache2/mod_version.so
0x10f550000 - 0x10f566fff mod_proxy.so (??? - ???) <0169F3B2-A81A-3E23-92FE-8E9B92C38795> /usr/libexec/apache2/mod_proxy.so
0x10f56e000 - 0x10f576ff7 mod_proxy_http.so (??? - ???) <F5B55152-D3D1-34AE-A95A-029F01476A54> /usr/libexec/apache2/mod_proxy_http.so
0x10f57c000 - 0x10f57efff mod_proxy_scgi.so (??? - ???) <DFD46BF1-F69D-383B-8B33-5273DD8037E4> /usr/libexec/apache2/mod_proxy_scgi.so
0x10f583000 - 0x10f589ff7 mod_proxy_balancer.so (??? - ???) <BC9C283D-8852-3016-8142-96A5675956FA> /usr/libexec/apache2/mod_proxy_balancer.so


I can post more details as requested.







apache-2.2 php httpd mac-osx-server process






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited May 31 '12 at 20:10







Thomas Adams

















asked May 26 '12 at 15:27









Thomas AdamsThomas Adams

243




243












  • I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

    – Stefan Lasiewski
    May 27 '12 at 18:13

















  • I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

    – Stefan Lasiewski
    May 27 '12 at 18:13
















I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

– Stefan Lasiewski
May 27 '12 at 18:13





I ran into a similar problem once on FreeBSD. See /server-status shows over 240 requests like “OPTIONS * HTTP/1.0” 200 - “-” “Apache (internal dummy connection)” . The trigger was filesystem snapshots, which momentarily suspended writes to the filesystem so that the system could take a filesystem snapshot. Is your filesystem healthy? Anything unusual on your filesystem?

– Stefan Lasiewski
May 27 '12 at 18:13










2 Answers
2






active

oldest

votes


















0














I am not totally familiar with the macosx process command line tools, but if those values in the 2nd column are httpd child process pids, then I would try and attach to the process using the dtrace tool and see what they are up to while hung like that.



dtrace -p 12345


http://www.troeger.eu/cms/?p=255
man pages






share|improve this answer























  • I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

    – Thomas Adams
    May 31 '12 at 20:07



















0














So I've solved this problem by rewriting some of the code. It appears this odd behavior was due to something in my PHP code. I don't know what it was exactly, but I removed a number of global variables and broke several parts into smaller functions, and the issue stopped. It's odd that PHP's timeout wouldn't be triggered, but there you go.






share|improve this answer























    Your Answer








    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "2"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f392937%2fapaches-options-http-1-0-running-100-cpu-runaway-httpd-process%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    I am not totally familiar with the macosx process command line tools, but if those values in the 2nd column are httpd child process pids, then I would try and attach to the process using the dtrace tool and see what they are up to while hung like that.



    dtrace -p 12345


    http://www.troeger.eu/cms/?p=255
    man pages






    share|improve this answer























    • I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

      – Thomas Adams
      May 31 '12 at 20:07
















    0














    I am not totally familiar with the macosx process command line tools, but if those values in the 2nd column are httpd child process pids, then I would try and attach to the process using the dtrace tool and see what they are up to while hung like that.



    dtrace -p 12345


    http://www.troeger.eu/cms/?p=255
    man pages






    share|improve this answer























    • I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

      – Thomas Adams
      May 31 '12 at 20:07














    0












    0








    0







    I am not totally familiar with the macosx process command line tools, but if those values in the 2nd column are httpd child process pids, then I would try and attach to the process using the dtrace tool and see what they are up to while hung like that.



    dtrace -p 12345


    http://www.troeger.eu/cms/?p=255
    man pages






    share|improve this answer













    I am not totally familiar with the macosx process command line tools, but if those values in the 2nd column are httpd child process pids, then I would try and attach to the process using the dtrace tool and see what they are up to while hung like that.



    dtrace -p 12345


    http://www.troeger.eu/cms/?p=255
    man pages







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered May 27 '12 at 17:41









    Tom HTom H

    8,56833257




    8,56833257












    • I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

      – Thomas Adams
      May 31 '12 at 20:07


















    • I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

      – Thomas Adams
      May 31 '12 at 20:07

















    I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

    – Thomas Adams
    May 31 '12 at 20:07






    I can sample the process from the activity monitor, and it appears to be endless non-repeating Apple code. I'll post some in my original question

    – Thomas Adams
    May 31 '12 at 20:07














    0














    So I've solved this problem by rewriting some of the code. It appears this odd behavior was due to something in my PHP code. I don't know what it was exactly, but I removed a number of global variables and broke several parts into smaller functions, and the issue stopped. It's odd that PHP's timeout wouldn't be triggered, but there you go.






    share|improve this answer



























      0














      So I've solved this problem by rewriting some of the code. It appears this odd behavior was due to something in my PHP code. I don't know what it was exactly, but I removed a number of global variables and broke several parts into smaller functions, and the issue stopped. It's odd that PHP's timeout wouldn't be triggered, but there you go.






      share|improve this answer

























        0












        0








        0







        So I've solved this problem by rewriting some of the code. It appears this odd behavior was due to something in my PHP code. I don't know what it was exactly, but I removed a number of global variables and broke several parts into smaller functions, and the issue stopped. It's odd that PHP's timeout wouldn't be triggered, but there you go.






        share|improve this answer













        So I've solved this problem by rewriting some of the code. It appears this odd behavior was due to something in my PHP code. I don't know what it was exactly, but I removed a number of global variables and broke several parts into smaller functions, and the issue stopped. It's odd that PHP's timeout wouldn't be triggered, but there you go.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jun 10 '12 at 17:51









        Thomas AdamsThomas Adams

        243




        243



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Server Fault!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f392937%2fapaches-options-http-1-0-running-100-cpu-runaway-httpd-process%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            How to write a 12-bar blues melodyI-IV-V blues progressionHow to play the bridges in a standard blues progressionHow does Gdim7 fit in C# minor?question on a certain chord progressionMusicology of Melody12 bar blues, spread rhythm: alternative to 6th chord to avoid finger stretchChord progressions/ Root key/ MelodiesHow to put chords (POP-EDM) under a given lead vocal melody (starting from a good knowledge in music theory)Are there “rules” for improvising with the minor pentatonic scale over 12-bar shuffle?Confusion about blues scale and chords

            What if the end-user didn't have the required library?What is setup.py?What is a clean, pythonic way to have multiple constructors in Python?What does Ruby have that Python doesn't, and vice versa?What is the reason for having '//' in Python?How do I create a namespace package in Python?How to package shared objects that python modules depend on?setuptools vs. distutils: why is distutils still a thing?Navigation in Windows 10 vs code not going to virtualenv library when the same library is installed at user levelPython create package for local usePackaging a project that uses multiple python versionsWhy is permission denied on pip install except for when “--user” is included at end of command?

            Esgonzo ibérico Índice Descrición Distribución Hábitat Ameazas Notas Véxase tamén "Acerca dos nomes dos anfibios e réptiles galegos""Chalcides bedriagai"Chalcides bedriagai en Carrascal, L. M. Salvador, A. (Eds). Enciclopedia virtual de los vertebrados españoles. Museo Nacional de Ciencias Naturales, Madrid. España.Fotos