Dec 26 19:15:32 ns365311 kernel: VM: killing process apache2
Dec 26 19:15:32 ns365311 kernel: swap_free: Unused swap offset entry 00000080
Dec 26 19:15:35 ns365311 last message repeated 33 times
Dec 26 19:24:06 ns365311 kernel: swap_free: Unused swap offset entry 00000080
Dec 26 19:24:06 ns365311 last message repeated 5 times
Dec 26 19:24:06 ns365311 kernel: Eeek! page_mapcount(page) went negative! (-1)
Dec 26 19:24:06 ns365311 kernel:   page pfn = 12253
Dec 26 19:24:06 ns365311 kernel:   page->flags = 10000000000003c
Dec 26 19:24:06 ns365311 kernel:   page->count = 1
Dec 26 19:24:06 ns365311 kernel:   page->mapping = ffff810125933538
Dec 26 19:24:06 ns365311 kernel:   vma->vm_ops = 0xffffffff804ed040
Dec 26 19:24:06 ns365311 kernel:   vma->vm_ops->nopage = _stext+0x7fdf7000/0x1a
Dec 26 19:24:06 ns365311 kernel:   vma->vm_ops->fault = shmem_fault+0x0/0x68
Dec 26 19:24:06 ns365311 kernel:   vma->vm_file->f_op->mmap = shmem_mmap+0x0/0x2e
Dec 26 19:24:06 ns365311 kernel: ------------[ cut here ]------------
Dec 26 19:24:06 ns365311 kernel: kernel BUG at mm/rmap.c:631!
Dec 26 19:24:06 ns365311 kernel: invalid opcode: 0000 [1] SMP 
Dec 26 19:24:06 ns365311 kernel: CPU 1 
Dec 26 19:24:06 ns365311 kernel: Modules linked in: drbd cn ac battery ip6table_filter ip6_tables iptable_raw xt_policy xt_multiport ipt_ULOG ipt_TTL ipt_ttl ipt_TOS ipt_tos ipt_SAME ipt_REJECT ipt_REDIRECT ipt_recent ipt_owner ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_iprange ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype xt_tcpmss xt_pkttype xt_physdev xt_NFQUEUE xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables ipv6 x_tables dm_snapshot dm_mirror dm_mod loop iTCO_wdt serio_raw pcspkr i2c_i801 psmouse i2c_core video output intel_agp button evdev ext3 jbd mbcache raid1 md_mod ide_generic sd_mod ata_generic ata_piix libata scsi_mod piix generic ide_core ehci_hcd uhci_hcd r8169 thermal processor fan
Dec 26 19:24:06 ns365311 kernel: Pid: 23398, comm: apache2 Not tainted 2.6.24-etchnhalf.1-amd64 #1
Dec 26 19:24:06 ns365311 kernel: RIP: 0010:[<ffffffff8028684c>]  [<ffffffff8028684c>] page_remove_rmap+0x104/0x120
Dec 26 19:24:06 ns365311 kernel: RSP: 0018:ffff810054a4bde8  EFLAGS: 00010292
Dec 26 19:24:06 ns365311 kernel: RAX: 0000000000000034 RBX: ffff81012c178228 RCX: ffffffff804e0a28
Dec 26 19:24:06 ns365311 kernel: RDX: 0000000100000000 RSI: 0000000000000092 RDI: ffffffff804e0a20
Dec 26 19:24:06 ns365311 kernel: RBP: ffff81007c0f61e8 R08: ffffffff804e0a10 R09: ffff810001025ee0
Dec 26 19:24:06 ns365311 kernel: R10: 0000000000000000 R11: ffffffff8021f8ee R12: 00002b4bae239000
Dec 26 19:24:06 ns365311 kernel: R13: ffff8100051fd1c8 R14: ffff810001029640 R15: 00002b4bae400000
Dec 26 19:24:06 ns365311 kernel: FS:  0000000000000000(0000) GS:ffff81012b67d9c0(0000) knlGS:0000000000000000
Dec 26 19:24:06 ns365311 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec 26 19:24:06 ns365311 kernel: CR2: 00002b4ba8818610 CR3: 0000000000201000 CR4: 00000000000006e0
Dec 26 19:24:06 ns365311 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 26 19:24:06 ns365311 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 26 19:24:06 ns365311 kernel: Process apache2 (pid: 23398, threadinfo ffff810054a4a000, task ffff81012972d000)
Dec 26 19:24:06 ns365311 kernel: Stack:  0000000000000206 ffff81012c178228 0000000012253020 ffffffff8027f499
Dec 26 19:24:06 ns365311 kernel:  0000000000000000 ffff810054a4bed8 ffffffffffffffff 0000000000000000
Dec 26 19:24:06 ns365311 kernel:  ffff81007c0f61e8 ffff810054a4bee0 000000000029779f 0000000000000000
Dec 26 19:24:06 ns365311 kernel: Call Trace:
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff8027f499>] unmap_vmas+0x420/0x743
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff80282e02>] exit_mmap+0x76/0xeb
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff80233495>] mmput+0x2c/0x9d
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff80238bc4>] do_exit+0x25f/0x796
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff8023917a>] sys_exit_group+0x0/0xe
Dec 26 19:24:06 ns365311 kernel:  [<ffffffff8020be2e>] system_call+0x7e/0x83
Dec 26 19:24:06 ns365311 kernel: 
Dec 26 19:24:06 ns365311 kernel: 
Dec 26 19:24:06 ns365311 kernel: Code: 0f 0b eb fe 8b 77 18 41 58 5b 5d 83 e6 01 f7 de 83 c6 04 e9 
Dec 26 19:24:06 ns365311 kernel: RIP  [<ffffffff8028684c>] page_remove_rmap+0x104/0x120
Dec 26 19:24:06 ns365311 kernel:  RSP <ffff810054a4bde8>
Dec 26 19:24:06 ns365311 kernel: ---[ end trace c2d77a0121baf934 ]---
Dec 26 19:24:06 ns365311 kernel: Fixing recursive fault but reboot is needed!
Dec 26 19:34:37 ns365311 kernel: swap_free: Unused swap offset entry 00000080
Dec 26 19:34:37 ns365311 last message repeated 13 times
Dec 26 20:06:22 ns365311 kernel: Unable to handle kernel NULL pointer dereference at 0000000000000028 RIP: 
Dec 26 20:06:22 ns365311 kernel:  [<0000000000000028>]
Dec 26 20:06:22 ns365311 kernel: PGD 6dce067 PUD b38a9067 PMD 0 
Dec 26 20:06:22 ns365311 kernel: Oops: 0010 [2] SMP 
Dec 26 20:06:22 ns365311 kernel: CPU 1 
Dec 26 20:06:22 ns365311 kernel: Modules linked in: drbd cn ac battery ip6table_filter ip6_tables iptable_raw xt_policy xt_multiport ipt_ULOG ipt_TTL ipt_ttl ipt_TOS ipt_tos ipt_SAME ipt_REJECT ipt_REDIRECT ipt_recent ipt_owner ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_iprange ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype xt_tcpmss xt_pkttype xt_physdev xt_NFQUEUE xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables ipv6 x_tables dm_snapshot dm_mirror dm_mod loop iTCO_wdt serio_raw pcspkr i2c_i801 psmouse i2c_core video output intel_agp button evdev ext3 jbd mbcache raid1 md_mod ide_generic sd_mod ata_generic ata_piix libata scsi_mod piix generic ide_core ehci_hcd uhci_hcd r8169 thermal processor fan
Dec 26 20:06:22 ns365311 kernel: Pid: 25310, comm: apache2 Tainted: G      D 2.6.24-etchnhalf.1-amd64 #1
Dec 26 20:06:22 ns365311 kernel: RIP: 0010:[<0000000000000028>]  [<0000000000000028>]
Dec 26 20:06:22 ns365311 kernel: RSP: 0000:ffff810005249d60  EFLAGS: 00010246
Dec 26 20:06:22 ns365311 kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff804e4860
Dec 26 20:06:22 ns365311 kernel: RDX: ffff810005249db4 RSI: 00002b4ba3a5c000 RDI: ffff8100051fcce8
Dec 26 20:06:22 ns365311 kernel: RBP: ffff8100c6d2e2e0 R08: 000000000000004e R09: 0000000000000000
Dec 26 20:06:22 ns365311 kernel: R10: 0000000000000000 R11: 00002b4ba4344da0 R12: ffff8100051fcce8
Dec 26 20:06:22 ns365311 kernel: R13: ffff8100052430c0 R14: 00002b4ba3a5c0f8 R15: 00002b4ba3a5c0f8
Dec 26 20:06:22 ns365311 kernel: FS:  00002b4ba56fb5c0(0000) GS:ffff81012b67d9c0(0000) knlGS:0000000000000000
Dec 26 20:06:22 ns365311 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 26 20:06:22 ns365311 kernel: CR2: 0000000000000028 CR3: 000000002682d000 CR4: 00000000000006e0
Dec 26 20:06:22 ns365311 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 26 20:06:22 ns365311 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 26 20:06:22 ns365311 kernel: Process apache2 (pid: 25310, threadinfo ffff810005248000, task ffff8100b6779800)
Dec 26 20:06:22 ns365311 kernel: Stack:  ffffffff8027e4b6 ffff8100c6d2e068 ffff810069f9b8e8 ffff81012e8fe220
Dec 26 20:06:22 ns365311 kernel:  ffffffff8027ec0b ffffffff00000000 000000000000004e 00002b4ba3a5c000
Dec 26 20:06:22 ns365311 kernel:  0000000000000000 0000000000000000 000000002e8fe220 0000000000000000
Dec 26 20:06:22 ns365311 kernel: Call Trace:
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff8027e4b6>] __do_fault+0x8c/0x39b
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff8027ec0b>] do_wp_page+0x446/0x4f1
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff80280055>] handle_mm_fault+0x330/0x6d8
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff802235ea>] do_page_fault+0x358/0x702
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff8024813f>] bit_waitqueue+0x1c/0x90
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff802a8008>] d_kill+0x44/0x59
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff802a82cb>] dput+0x1c/0x10b
Dec 26 20:06:22 ns365311 kernel:  [<ffffffff80415f79>] error_exit+0x0/0x51
Dec 26 20:06:22 ns365311 kernel: 
Dec 26 20:06:22 ns365311 kernel: 
Dec 26 20:06:22 ns365311 kernel: Code:  Bad RIP value.
Dec 26 20:06:22 ns365311 kernel: RIP  [<0000000000000028>]
Dec 26 20:06:22 ns365311 kernel:  RSP <ffff810005249d60>
Dec 26 20:06:22 ns365311 kernel: CR2: 0000000000000028
Dec 26 20:06:22 ns365311 kernel: ---[ end trace c2d77a0121baf934 ]---
Dec 26 20:06:30 ns365311 kernel: general protection fault: 0000 [3] SMP 
Dec 26 20:06:30 ns365311 kernel: CPU 1 
Dec 26 20:06:30 ns365311 kernel: Modules linked in: drbd cn ac battery ip6table_filter ip6_tables iptable_raw xt_policy xt_multiport ipt_ULOG ipt_TTL ipt_ttl ipt_TOS ipt_tos ipt_SAME ipt_REJECT ipt_REDIRECT ipt_recent ipt_owner ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_iprange ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype xt_tcpmss xt_pkttype xt_physdev xt_NFQUEUE xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables ipv6 x_tables dm_snapshot dm_mirror dm_mod loop iTCO_wdt serio_raw pcspkr i2c_i801 psmouse i2c_core video output intel_agp button evdev ext3 jbd mbcache raid1 md_mod ide_generic sd_mod ata_generic ata_piix libata scsi_mod piix generic ide_core ehci_hcd uhci_hcd r8169 thermal processor fan
Dec 26 20:06:30 ns365311 kernel: Pid: 25310, comm: apache2 Tainted: G      D 2.6.24-etchnhalf.1-amd64 #1
Dec 26 20:06:30 ns365311 kernel: RIP: 0010:[<ffffffff803111a2>]  [<ffffffff803111a2>] get_index+0xe/0x3f
Dec 26 20:06:30 ns365311 kernel: RSP: 0000:ffff810005249a98  EFLAGS: 00010202
Dec 26 20:06:30 ns365311 kernel: RAX: 203fb8f645a0f0be RBX: ffff810074c83700 RCX: ffff810005249aa8
Dec 26 20:06:30 ns365311 kernel: RDX: 762d6568636162e7 RSI: 762d65686361632f RDI: ffff810042176240
Dec 26 20:06:30 ns365311 kernel: RBP: ffff810042176240 R08: ffff810005249ab8 R09: 0000000000a4cfff
Dec 26 20:06:30 ns365311 kernel: R10: 0000000000a4cfff R11: 0000000000000001 R12: ffff810005249ab8
Dec 26 20:06:30 ns365311 kernel: R13: ffff8100051fcf40 R14: 0000000000000000 R15: 0000000000000000
Dec 26 20:06:30 ns365311 kernel: FS:  0000000000000000(0000) GS:ffff81012b67d9c0(0000) knlGS:0000000000000000
Dec 26 20:06:30 ns365311 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 26 20:06:30 ns365311 kernel: CR2: 0000000000000028 CR3: 0000000000201000 CR4: 00000000000006e0
Dec 26 20:06:30 ns365311 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 26 20:06:30 ns365311 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 26 20:06:30 ns365311 kernel: Process apache2 (pid: 25310, threadinfo ffff810005248000, task ffff8100b6779800)
Dec 26 20:06:30 ns365311 kernel: Stack:  ffffffff8031141e 0000000000a4d000 000000000000001f 203fb8f645a0f0be
Dec 26 20:06:30 ns365311 kernel:  2030393633203030 ffff810042176220 ffff81004b5b1e40 ffff8100051fcef8
Dec 26 20:06:30 ns365311 kernel:  00002b4ba2fbd000 ffffffff8028320d ffff8100051fce48 ffff8100051fcef8
Dec 26 20:06:30 ns365311 kernel: Call Trace:
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8031141e>] prio_tree_remove+0x2a/0xbc
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8028320d>] unlink_file_vma+0x31/0x3d
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80281ace>] free_pgtables+0x74/0xc4
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80282e1d>] exit_mmap+0x91/0xeb
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80233495>] mmput+0x2c/0x9d
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80238bc4>] do_exit+0x25f/0x796
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8020d034>] oops_end+0x33/0x50
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff802238bd>] do_page_fault+0x62b/0x702
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff880fb51a>] :jbd:journal_stop+0x191/0x19d
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8027cfd2>] zone_statistics+0x3f/0x60
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80415f79>] error_exit+0x0/0x51
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8027e4b6>] __do_fault+0x8c/0x39b
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8027ec0b>] do_wp_page+0x446/0x4f1
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80280055>] handle_mm_fault+0x330/0x6d8
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff802235ea>] do_page_fault+0x358/0x702
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff8024813f>] bit_waitqueue+0x1c/0x90
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff802a8008>] d_kill+0x44/0x59
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff802a82cb>] dput+0x1c/0x10b
Dec 26 20:06:30 ns365311 kernel:  [<ffffffff80415f79>] error_exit+0x0/0x51
Dec 26 20:06:30 ns365311 kernel: 
Dec 26 20:06:30 ns365311 kernel: 
Dec 26 20:06:30 ns365311 kernel: Code: 48 8b 82 88 00 00 00 49 89 00 48 8b 46 c8 48 2b 46 c0 48 c1 
Dec 26 20:06:30 ns365311 kernel: RIP  [<ffffffff803111a2>] get_index+0xe/0x3f
Dec 26 20:06:30 ns365311 kernel:  RSP <ffff810005249a98>
Dec 26 20:06:30 ns365311 kernel: ---[ end trace c2d77a0121baf934 ]---
Dec 26 20:06:30 ns365311 kernel: Fixing recursive fault but reboot is needed!