EzDevInfo.com

Relationship between a kernel and a user thread

Is there a relationship between a kernel and a user thread?

Some operating system textbooks said that "maps one (many) user thread to one (many) kernel thread". What does map means here?

What is the difference between the kernel space and the user space?

What is the difference between the kernel space and the user space? Do kernel space, kernel threads, kernel processes and kernel stack mean the same thing? Also, why do we need this differentiation?

Learning Kernel Programming [closed]

I want to learn linux kernel programming.

What would be the starting points for that ? What could be some of the simpler problems to target ?

thanks in advance

Learning kernel hacking and embedded development at home? [closed]

I was always attracted to the world of kernel hacking and embedded systems.
Has anyone got good tutorials (+easily available hardware) on starting to mess with such stuff?
Something like kits for writing drivers etc, which come with good documentation and are affordable?

Thanks!

What IDE would be good for linux kernel driver development [closed]

I am using ubuntu 8.04.1 and i am trying to write a character driver in kernel mode. What would be a good ide, ideally with code completion, to do that ?

What are some resources for getting started in operating system development? [closed]

One thing I've always wanted to do is develop my very own operating system (not necessarily fancy like Linux or Windows, but better than a simple boot loader which I've already done).

I'm having a hard time finding resources/guides that take you past writing a simple "Hello World" OS.

I know lots of people will probably recommend I look at Linux or BSD; but the code base for systems like that is (presumably) so big that I wouldn't know where to start.

Any suggestions?

Update: To make it easier for people who land on this post through Google here are some OS development resources:

• Writing Your Own Operating System (Thanks Adam)

• Linux From Scratch (Thanks John)

• SharpOS (C# Operating System) (Thanks lomaxx)

• Minix3 and Minix2 (Thanks Mike)

• OS Dev Wiki and Forums (Thanks Steve)

• BonaFide (Thanks Steve)

• Bran (Thanks Steve)

• Roll your own toy UNIX-clone OS (Thanks Steve)

• Broken Thorn OS Development Series

Other resources:

I found a nice resource named MikeOS, "MikeOS is a learning tool to demonstrate how simple OSes work. It uses 16-bit real mode for BIOS access, so that it doesn't need complex drivers"

Updated 11/14/08

I found some resources at Freebyte's Guide to...Free and non-free Operating Systems that links to kits such as OSKit and ExOS library. These seem super useful in getting started in OS development.

Updated 2/23/09

Ric Tokyo recommended nanoos in this question. Nanoos is an OS written in C++.

Updated 3/9/09

Dinah provided some useful Stack Overflow discussion of aspiring OS developers: Roadblocks in creating a custom operating system discusses what pitfalls you might encounter while developing an OS and OS Development is a more general discussion.

Updated 7/9/09

LB provided a link to the Pintos Project, an education OS designed for students learning OS development.

Updated 7/27/09 (Still going strong!)

I stumbled upon an online OS course from Berkley featuring 23 lectures.

TomOS is a fork of MikeOS that includes a little memory manager and mouse support. As MikeOS, it is designed to be an educational project. It is written in NASM assembler.

Updated 8/4/09

I found the slides and other materials to go along with the online Berkeley lectures listed above.

Updated 8/23/09

All questions tagged osdev on stackoverflow

OS/161 is an academic OS written in c that runs on a simulated hardware. This OS is similar in Nachos. Thanks Novelocrat!

tangurena recommends http://en.wikipedia.org/wiki/MicroC/OS-II, an OS designed for embedded systems. There is a companion book as well.

Linux Kernel Development by Robert Love is suggested by Anders. It is a "widely acclaimed insider's look at the Linux kernel."

Updated 9/18/2009

Thanks Tim S. Van Haren for telling us about Cosmos, an OS written entirely in c#.

tgiphil tells us about Managed Operating System Alliance (MOSA) Framework, "a set of tools, specifications and source code to foster development of managed operating systems based on the Common Intermediate Language."

Update 9/24/2009

Steve found a couple resources for development on windows using Visual Studio, check out BrokenThorn's guide setup with VS 2005 or OSDev's VS Section.

Update 1/20/2012

A set of tutorials aims to take you through programming a simple UNIX-clone operating system for the x86 architecture. JamesM's kernel development tutorials

Updated 9/5/2012

kerneltrap.org is no longer available. The linux kernel v0.01 is available from kernel.org

Updated 12/21/2012 A basic OS development tutorial designed to be a semester's project. It guides you through to build an OS with basic components. Very good start for beginners. Related paper. Thanks Srujan!

Updated 11/15/2013

Writing a Simple Operating System From Scratch. Thanks James Moore!

Updated 12/8/2013

How to make a computer operating system Thanks ddtoni!

Updated 3/18/2014

ToAruOS an OS built mostly from scratch, including GUI

cscope or ctags why choose one over the other?

I primarily use vim / gvim as an editor and am looking at using a combination of lxr (the Linux Cross Reference) and either cscope or ctags for exploring the kernel source. However, I haven't ever used either cscope or ctags and would like to hear why one might choose one over the other taking into consideration my use of vim as a primary editor.

What is an OS kernel ? How does it differ from an operating system? [closed]

I am not able to understand the difference between a kernel and an operating system. I do not see any difference between them. Is the kernel an operating system?

What is this style of syntax in C?

From sys.c line 123:

void *sys_call_table[__NR_syscalls] = 
{
    [0 ... __NR_syscalls-1] = sys_ni_syscall,
#include <asm/unistd.h>
};

sys_call_table is a generic pointer to arrays, I can see that. However what is the notation:

[0 ... __NR_syscalls-1]

What is the ...?

EDIT:
I learned another C trick here: #include <asm/unistd.h> will be preprocessed and replaced with its content and assigned to [0 ... _NR_syscalls-1].

What is the difference between vmalloc and kmalloc?

I've googled around and found most people advocating the use of kmalloc, as you're guaranteed to get contiguous physical blocks of memory. However, it also seems as though kmalloc can fail if a contiguous physical block that you want can't be found.
What are the advantages of having a contiguous block of memory? Specifically, why would I need to have a contiguous physical block of memory in a system call? Is there any reason I couldn't just use vmalloc?
Finally, if I were to allocate memory during the handling of a system call, should I specify GFP_ATOMIC? Is a system call executed in an atomic context?

GFP_ATOMIC
The allocation is high-priority and does not sleep. This is the flag to use in interrupt handlers, bottom halves and other situations where you cannot sleep.

GFP_KERNEL This is a normal allocation and might block. This is the flag to use in process context code when it is safe to sleep.

Context switches much slower in new linux kernels

We are looking to upgrade the OS on our servers from Ubuntu 10.04 LTS to Ubuntu 12.04 LTS. Unfortunately, it seems that the latency to run a thread that has become runnable has significantly increased from the 2.6 kernel to the 3.2 kernel. In fact the latency numbers we are getting are hard to believe.

Let me be more specific about the test. We have a program that runs two threads. The first thread gets the current time (in ticks using RDTSC) and then signals a condition variable once a second. The second thread waits on the condition variable and wakes up when it is signaled. It then gets the current time (in ticks using RDTSC). The difference between the time in the second thread and the time in the first thread is computed and displayed on the console. After this the second thread waits on the condition variable once more. It will be signaled again by the first thread after about a second passes.

So, in a nutshell we get a thread to thread communication via condition variable latency measurement once a second as a result.

In kernel 2.6.32, this latency is somewhere on the order of 2.8-3.5 us, which is reasonable. In kernel 3.2.0, this latency has increased to somewhere on the order of 40-100 us. I have excluded any differences in hardware between the two hosts. They run on identical hardware (dual socket X5687 {Westmere-EP} processors running at 3.6 GHz with hyperthreading, speedstep and all C states turned off). The test app changes the affinity of the threads to run them on independent physical cores of the same socket (i.e., the first thread is run on Core 0 and the second thread is run on Core 1), so there is no bouncing of threads on cores or bouncing/communication between sockets.

The only difference between the two hosts is that one is running Ubuntu 10.04 LTS with kernel 2.6.32-28 (the fast context switch box) and the other is running the latest Ubuntu 12.04 LTS with kernel 3.2.0-23 (the slow context switch box). All BIOS settings and hardware are identical.

Have there been any changes in the kernel that could account for this ridiculous slow down in how long it takes for a thread to be scheduled to run?

Update: If you would like to run the test on your host and linux build, I have posted the code to pastebin for your perusal. Compile with:

g++ -O3 -o test_latency test_latency.cpp -lpthread

Run with (assuming you have at least a dual-core box):

./test_latency 0 1 # Thread 1 on Core 0 and Thread 2 on Core 1

Update 2: After much searching through kernel parameters, posts on kernel changes and personal research, I have figured out what the problem is and have posted the solution as an answer to this question.

What means the dot before variable name in struct?

looking at the linux kernel source, I found this:

static struct tty_operations serial_ops = {
  .open = tiny_open,
  .close = tiny_close,
  .write = tiny_write,
  .write_room = tiny_write_room,
  .set_termios = tiny_set_termios,
};

I've never seen such a notation in C. Why is there a dot before the variable name?

What is __gxx_personality_v0 for?

This is a second-hand question from an OS development site, but it made me curious since I couldn't find a decent explanation anywhere.

When compiling and linking a free-standing C++ program using gcc, sometimes a linker error like this occurs:

out/kernel.o:(.eh_frame+0x11): undefined reference to `__gxx_personality_v0'

This is apparently because this symbol is defined in libstdc++, which is missing in a free-standing environment. Fixing the problem simply requires defining this symbol somewhere:

void *__gxx_personality_v0;

Which is nice, but I don't like things that just magically work... So the question is, what is the purpose of this symbol?

Linux Process States

In Linux, what happens to the state of a process when it needs to read blocks from a disk? Is it blocked? If so, how is another process chosen to execute?

What kind of C is an operating system written in?

It makes sense that something like an operating system would be written in C. But how much of it, and what kind of C? I mean, in C, if you needed some heap memory, you would call malloc. But, does an OS even have a heap? As far as I know, malloc asks the operating system for memory and then adds it to a linked list, or binary tree, or something. What about a call stack? The OS is responsible for setting up all of this stuff that other applications use, but how does it do that? When you want to open or create a file in C, the appropriate functions ask the operating system for that file. so... What kind of C is on the other side of that call? Or on the other end of a memory allocation?

Also, how much of an operating system would actually be written in C? All of it? What about architecture dependent code? What about the higher levels of abstraction--does that ever get written in higher level languages, like C++?

I mean, I'm just asking this out of sheer curiosity. I'm downloading the latest linux kernel now but it's taking forever. I'm not sure if I'll wind up being able to follow the code--or if I'll be caught in an inescapably complex web of stuff I've never seen before.