EzDevInfo.com

Increasing the maximum number of tcp/ip connections in linux

I am programming a server and it seems like my number of connections is being limited since my bandwidth isn't being saturated even when I've set the number of connections to "unlimited".

How can I increase or eliminate a maximum number of connections that my Ubuntu Linux box can open at a time? Does the OS limit this, or is it the router or the ISP? Or is it something else?

Why is Linux called a monolithic kernel?

I read that Linux is a monolithic kernel. Does monolithic kernel mean compiling and linking the complete kernel code into an executable?

If Linux is able to support modules, why not break all the subsystems into modules and load them when necessary? In that case, the kernel doesn't have to load all modules initially and could maintain an index of the functions in the module and load them when necessary.

Is bool a native C type?

I've noticed that the Linux kernel code uses bool, but I thought that bool was a C++ type. Is bool a standard C extension (e.g., ISO C90) or a GCC extension?

What's the use of do while(0) when we define a macro? [duplicate]

Possible Duplicate:
Do-While and if-else statements in C/C++ macros

I'm reading the linux kernel and I found many macros like this:

#define INIT_LIST_HEAD(ptr) do { \
    (ptr)->next = (ptr); (ptr)->prev = (ptr); \
} while (0)

Why do they use this rather than define it simply in a {}?

What is the theoretical maximum number of open TCP connections that a modern Linux box can have

Assuming infinite performance from hardware, can a Linux box support >65536 open TCP connections?

I understand that the number of ephemeral ports (<65536) limits the number of connections from one local IP to one port on one remote IP.

The tuple (local ip, local port, remote ip, remote port) is what uniquely defines a TCP connection; does this imply that more than 65K connections can be supported if more than one of these parameters are free. e.g. connections to a single port number on multiple remote hosts from multiple local IPs.

Is there another 16 bit limit in the system? Number of file descriptors perhaps?

What is ":-!!" in C code?

I bumped into this strange macro code in /usr/include/linux/kernel.h:

/* Force a compilation error if condition is true, but also produce a
   result (of value 0 and type size_t), so the expression can be used
   e.g. in a structure initializer (or where-ever else comma expressions
   aren't permitted). */
#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
#define BUILD_BUG_ON_NULL(e) ((void *)sizeof(struct { int:-!!(e); }))

What does :-!! do?

Why kernel code/thread executing in interrupt context cannot sleep?

I am reading following article by Robert Love

http://www.linuxjournal.com/article/6916

that says

"...Let's discuss the fact that work queues run in process context. This is in contrast to the other bottom-half mechanisms, which all run in interrupt context. Code running in interrupt context is unable to sleep, or block, because interrupt context does not have a backing process with which to reschedule. Therefore, because interrupt handlers are not associated with a process, there is nothing for the scheduler to put to sleep and, more importantly, nothing for the scheduler to wake up..."

I don't get it. AFAIK, scheduler in the kernel is O(1), that is implemented through the bitmap. So what stops the scehduler from putting interrupt context to sleep and taking next schedulable process and passing it the control?

Qt Creator, ptrace: Operation not permitted. What is the permanent solution?

While debugging C++ code in Qt creator I get the following error

ptrace: Operation not permitted.

Could not attach to the process. Make sure no other debugger traces this process.
Check the settings of
/proc/sys/kernel/yama/ptrace_scope
For more details, see /etc/sysctl.d/10-ptrace.conf

Here a temporary solution is found: Receiving error while trying to debug in QtProject

• temporary solution (won't survive a reboot):

  echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

But it is difficult to run the same code in terminal every time when I start my PC to use Qt.

What is the permanent solution for this?

How does Linux Kernel know where to look for driver firmware?

I'm compiling a custom kernel under Ubuntu and I'm running into the problem that my kernel doesn't seem to know where to look for firmware. Under Ubuntu 8.04, firmware is tied to kernel version the same way driver modules are. For example, kernel 2.6.24-24-generic stores its kernel modules in:

/lib/modules/2.6.24-24-generic

and its firmware in:

/lib/firmware/2.6.24-24-generic

When I compile the 2.6.24-24-generic Ubuntu kernel according the "Alternate Build Method: The Old-Fashioned Debian Way" I get the appropriate modules directory and all my devices work except those requiring firmware such as my Intel wireless card (ipw2200 module).

The kernel log shows for example that when ipw2200 tries to load the firmware the kernel subsystem controlling the loading of firmware is unable to locate it:

ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200: ipw2200-bss.fw request_firmware failed: Reason -2

errno-base.h defines this as:

#define ENOENT       2  /* No such file or directory */

(The function returning ENOENT puts a minus in front of it.)

I tried creating a symlink in /lib/firmware where my kernel's name pointed to the 2.6.24-24-generic directory, however this resulted in the same error. This firmware is non-GPL, provided by Intel and packed by Ubuntu. I don't believe it has any actual tie to a particular kernel version. cmp shows that the versions in the various directories are identical.

So how does the kernel know where to look for firmware?

Update

I found this solution to the exact problem I'm having, however it no longer works as Ubuntu has eliminated /etc/hotplug.d and no longer stores its firmware in /usr/lib/hotplug/firmware.

Update2

Some more research turned up some more answers. Up until version 92 of udev, the program firmware_helper was the way firmware got loaded. Starting with udev 93 this program was replaced with a script named firmware.sh providing identical functionality as far as I can tell. Both of these hardcode the firmware path to /lib/firmware. Ubuntu still seems to be using the /lib/udev/firmware_helper binary.

The name of the firmware file is passed to firmware_helper in the environment variable $FIRMWARE which is concatenated to the path /lib/firmware and used to load the firmware.

The actual request to load the firmware is made by the driver (ipw2200 in my case) via the system call:

request_firmware(..., "ipw2200-bss.fw", ...);

Now somewhere in between the driver calling request_firmware and firmware_helper looking at the $FIRMWARE environment variable, the kernel package name is getting prepended to the firmware name.

So who's doing it?

Content for Linux Operating Systems Class

I will be TA for an operating systems class this upcoming semester. The labs will deal specifically with the Linux Kernel.

1. What concepts/components of the Linux kernel do you think are the most important to cover in the class?
2. What do you wish was covered in your studies that was left out?

Any suggestions regarding the Linux kernel or overall operating systems design would be much appreciated.

kernel stack and user space stack

What's the difference between kernel stack and user stack? Why kernel stack is used? If a local variable is declared in an ISR, where it will be stored? Does each process has its own kernel stack ? Then how the process coordinates between both these stacks?

Direct Memory Access in Linux

I'm trying to access physical memory directly for an embedded Linux project, but I'm not sure how I can best designate memory for my use.

If I boot my device regularly, and access /dev/mem, I can easily read and write to just about anywhere I want. However, in this, I'm accessing memory that can easily be allocated to any process; which I don't want to do

My code for /dev/mem is (all error checking, etc. removed):

mem_fd = open("/dev/mem", O_RDWR));
mem_p = malloc(SIZE + (PAGE_SIZE - 1));
if ((unsigned long) mem_p % PAGE_SIZE) {
    mem_p += PAGE_SIZE - ((unsigned long) mem_p % PAGE_SIZE);
}
mem_p = (unsigned char *) mmap(mem_p, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, mem_fd, BASE_ADDRESS);

And this works. However, I'd like to be using memory that no one else will touch. I've tried limiting the amount of memory that the kernel sees by booting with mem=XXXm, and then setting BASE_ADDRESS to something above that (but below the physical memory), but it doesn't seem to be accessing the same memory consistently.

Based on what I've seen online, I suspect I may need a kernel module (which is OK) which uses either ioremap() or remap_pfn_range() (or both???), but I have absolutely no idea how; can anyone help?

EDIT: What I want is a way to always access the same physical memory (say, 1.5MB worth), and set that memory aside so that the kernel will not allocate it to any other process.

I'm trying to reproduce a system we had in other OSes (with no memory management) whereby I could allocate a space in memory via the linker, and access it using something like

*(unsigned char *)0x12345678

EDIT2: I guess I should provide some more detail. This memory space will be used for a RAM buffer for a high performance logging solution for an embedded application. In the systems we have, there's nothing that clears or scrambles physical memory during a soft reboot. Thus, if I write a bit to a physical address X, and reboot the system, the same bit will still be set after the reboot. This has been tested on the exact same hardware running VxWorks (this logic also works nicely in Nucleus RTOS and OS20 on different platforms, FWIW). My idea was to try the same thing in Linux by addressing physical memory directly; therefore, it's essential that I get the same addresses each boot.

I should probably clarify that this is for kernel 2.6.12 and newer.

EDIT3: Here's my code, first for the kernel module, then for the userspace application.

To use it, I boot with mem=95m, then insmod foo-module.ko, then mknod mknod /dev/foo c 32 0, then run foo-user , where it dies. Running under gdb shows that it dies at the assignment, although within gdb, I cannot dereference the address I get from mmap (although printf can)

foo-module.c

#include <linux/module.h>
#include <linux/config.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <asm/io.h>

#define VERSION_STR "1.0.0"
#define FOO_BUFFER_SIZE (1u*1024u*1024u)
#define FOO_BUFFER_OFFSET (95u*1024u*1024u)
#define FOO_MAJOR 32
#define FOO_NAME "foo"

static const char *foo_version = "@(#) foo Support version " VERSION_STR " " __DATE__ " " __TIME__;

static void    *pt = NULL;

static int      foo_release(struct inode *inode, struct file *file);
static int      foo_open(struct inode *inode, struct file *file);
static int      foo_mmap(struct file *filp, struct vm_area_struct *vma);

struct file_operations foo_fops = {
    .owner = THIS_MODULE,
    .llseek = NULL,
    .read = NULL,
    .write = NULL,
    .readdir = NULL,
    .poll = NULL,
    .ioctl = NULL,
    .mmap = foo_mmap,
    .open = foo_open,
    .flush = NULL,
    .release = foo_release,
    .fsync = NULL,
    .fasync = NULL,
    .lock = NULL,
    .readv = NULL,
    .writev = NULL,
};

static int __init foo_init(void)
{
    int             i;
    printk(KERN_NOTICE "Loading foo support module\n");
    printk(KERN_INFO "Version %s\n", foo_version);
    printk(KERN_INFO "Preparing device /dev/foo\n");
    i = register_chrdev(FOO_MAJOR, FOO_NAME, &foo_fops);
    if (i != 0) {
        return -EIO;
        printk(KERN_ERR "Device couldn't be registered!");
    }
    printk(KERN_NOTICE "Device ready.\n");
    printk(KERN_NOTICE "Make sure to run mknod /dev/foo c %d 0\n", FOO_MAJOR);
    printk(KERN_INFO "Allocating memory\n");
    pt = ioremap(FOO_BUFFER_OFFSET, FOO_BUFFER_SIZE);
    if (pt == NULL) {
        printk(KERN_ERR "Unable to remap memory\n");
        return 1;
    }
    printk(KERN_INFO "ioremap returned %p\n", pt);
    return 0;
}
static void __exit foo_exit(void)
{
    printk(KERN_NOTICE "Unloading foo support module\n");
    unregister_chrdev(FOO_MAJOR, FOO_NAME);
    if (pt != NULL) {
        printk(KERN_INFO "Unmapping memory at %p\n", pt);
        iounmap(pt);
    } else {
        printk(KERN_WARNING "No memory to unmap!\n");
    }
    return;
}
static int foo_open(struct inode *inode, struct file *file)
{
    printk("foo_open\n");
    return 0;
}
static int foo_release(struct inode *inode, struct file *file)
{
    printk("foo_release\n");
    return 0;
}
static int foo_mmap(struct file *filp, struct vm_area_struct *vma)
{
    int             ret;
    if (pt == NULL) {
        printk(KERN_ERR "Memory not mapped!\n");
        return -EAGAIN;
    }
    if ((vma->vm_end - vma->vm_start) != FOO_BUFFER_SIZE) {
        printk(KERN_ERR "Error: sizes don't match (buffer size = %d, requested size = %lu)\n", FOO_BUFFER_SIZE, vma->vm_end - vma->vm_start);
        return -EAGAIN;
    }
    ret = remap_pfn_range(vma, vma->vm_start, (unsigned long) pt, vma->vm_end - vma->vm_start, PAGE_SHARED);
    if (ret != 0) {
        printk(KERN_ERR "Error in calling remap_pfn_range: returned %d\n", ret);
        return -EAGAIN;
    }
    return 0;
}
module_init(foo_init);
module_exit(foo_exit);
MODULE_AUTHOR("Mike Miller");
MODULE_LICENSE("NONE");
MODULE_VERSION(VERSION_STR);
MODULE_DESCRIPTION("Provides support for foo to access direct memory");

foo-user.c

#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/mman.h>

int main(void)
{
    int             fd;
    char           *mptr;
    fd = open("/dev/foo", O_RDWR | O_SYNC);
    if (fd == -1) {
        printf("open error...\n");
        return 1;
    }
    mptr = mmap(0, 1 * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, 4096);
    printf("On start, mptr points to 0x%lX.\n",(unsigned long) mptr);
    printf("mptr points to 0x%lX. *mptr = 0x%X\n", (unsigned long) mptr, *mptr);
    mptr[0] = 'a';
    mptr[1] = 'b';
    printf("mptr points to 0x%lX. *mptr = 0x%X\n", (unsigned long) mptr, *mptr);
    close(fd);
    return 0;
}

min macro in kernel.h

In kernel.h min is defined as:

#define min(x, y) ({                \
    typeof(x) _min1 = (x);          \
    typeof(y) _min2 = (y);          \
    (void) (&_min1 == &_min2);      \
    _min1 < _min2 ? _min1 : _min2; })

I don't understand what the line (void) (&_min1 == &_min2); does. Is it some kind of type checking or something?

How does the linux kernel manage less than 1GB physical memory?

I'm learning the linux kernel internals and while reading "Understanding Linux Kernel", quite a few memory related questions struck me. One of them is, how the Linux kernel handles the memory mapping if the physical memory of say only 512 MB is installed on my system.

As I read, kernel maps 0(or 16) MB-896MB physical RAM into 0xC0000000 linear address and can directly address it. So, in the above described case where I only have 512 MB:

• How can the kernel map 896 MB from only 512 MB ? In the scheme described, the kernel set things up so that every process's page tables mapped virtual addresses from 0xC0000000 to 0xFFFFFFFF (1GB) directly to physical addresses from 0x00000000 to 0x3FFFFFFF (1GB). But when I have only 512 MB physical RAM, how can I map, virtual addresses from 0xC0000000-0xFFFFFFFF to physical 0x00000000-0x3FFFFFFF ? Point is I have a physical range of only 0x00000000-0x20000000.

• What about user mode processes in this situation?

• Every article explains only the situation, when you've installed 4 GB of memory and the kernel maps the 1 GB into kernel space and user processes uses the remaining amount of RAM.

I would appreciate any help in improving my understanding.

Thanks..!

Use of floating point in the Linux kernel

I am reading Robert Love's "Linux Kernel Development", and I came across the following passage:

No (Easy) Use of Floating Point

When a user-space process uses floating-point instructions, the kernel manages the transition from integer to floating point mode. What the kernel has to do when using floating-point instructions varies by architecture, but the kernel normally catches a trap and then initiates the transition from integer to floating point mode.

Unlike user-space, the kernel does not have the luxury of seamless support for floating point because it cannot easily trap itself. Using a floating point inside the kernel requires manually saving and restoring the floating point registers, among other possible chores. The short answer is: Don’t do it! Except in the rare cases, no floating-point operations are in the kernel.

I've never heard of these "integer" and "floating-point" modes. What exactly are they, and why are they needed? Does this distinction exist on mainstream hardware architectures (such as x86), or is it specific to some more exotic environments? What exactly does a transition from integer to floating point mode entail, both from the point of view of the process and the kernel?