Showing posts with label kernel. Show all posts
Showing posts with label kernel. Show all posts

Linux kernel understanding

The Ottawa LUG ( Linux Users Group ) created several videos that go into great detail about how the kernel is designed, structured, maintained, and used. It is reasonably informative and isn't too technical that it bogs down. The subjects are very complex and if you get side tracked into SCSI protocol, USB stacks, drivers and specifics of certain machine it can be years worth of information. These are deep enough to be informative and leave the technical detail to be known already or discovered, if needed.

One interesting thing I learned is that the kernel is "Object Flavored" in its implementation. C is not inherently OO ( Object Oriented ) and it is styled to be OO. This is exactly what I do in my code in a slightly different way as I define data sets that might be simple data types or complex structures or even sections of a structure to be "associated" with a method in context. The implementation in kernel and C++ is strictly single context association and I use a method that operates across data sets and then determines if a method is valid in a specific context, or excluded in a specific condition or context.

Another method I use is to funnel all calls through a single point. It distributes those system calls in categories and sections. The advantage of this is debugging , and I use a logging debugger which has a circular buffer to track the code. By placing my own break codes and 'int 3' in the code and the ability to control the level of debug it allows a much faster resolution of complex problems that may depend on what conditions transpired in parallel or previous to the fault. I also track each hardware interrupt on a conditional that is adjustable in the variables page.

Another advantage of centralizing system function calls is that I can essentially throttle the speed of the code to make it slower and slower as it approaches a point which is of interest. This is primarily for user space code that faults and then allows me to switch to debug or forces me to debug on a critical fault. It is not always necessary, however some faults are not immediately obvious, even to a trained programmer.

Another advantage of this debugger is that it will run code as symbolic. I can set up a virtual machine and pretend to do the operations in a symbolic context. I use variables like R10 as register 10 and simply have a distributor table that matches the machine code to a simulation of the code. The most difficult part of that is EA ( Effective Address ). If Virtual Effective Address ( Simulating a process running on a protected mode machine) is included it becomes almost mind numbingly complex. This also allows me to run code without a physical interface or debug virus like code in a safe way that produces list of what it did with which pseudo resources.

IBM also has much documentation on Linux and that is commendable. IBM is another bygone monopoly that has seen the light ( somewhat ). In their 'time of monopoly' they used their position to do some questionable things also.

Google has managed to stay away from the pitfalls of a monopoly and that is commendable. I think it is because they were aware that it is a company killer and it is not really good for people in general. I am not familiar with the founders, but I do hear the phrase "Do no evil." mentioned as a corporate policy.

ADDED: It is very interesting after going through the detail of how the Linux kernel initializes. If I were paranoid, I would think they had my code to use as a template. The truth is that there are not many ways that this can be done and be stable and proper. The order of events is determined by the hardware. Except for the fact that I used assembly right up to the point of the switch to protected mode, the code is the same. I can read the assembly and even the C as easily. Some of the features of GCC are new to me and some of the terminology is different, but the concepts have to be implemented a certain way. I do mutex ( mutual exclusion ) and deadly embrace avoidance, but I never called them semaphores. This highlights the fact that the type of activities that Ms has been involved in are completely inappropriate. Any person has to design the code around certain methods and by patenting or roadblocking a specific method that is required to achieve a goal is the same as poisoning everyone to benefit oneself.

This is one of those cases where BIOS experience, assembly(ATT & Intel), pmode structure, and hardware experience make this almost a walk in the park. I should have my own version of the kernel with a patch file in a couple days. The biggest gain that I see in that is the fact that I have a ram list based debugger that is dead solid and monumental in its capability. It works seamlessly in real and protected mode and that is very tricky. The absolute necessity is to never have a second fault in debug, as it is two strikes and you're out! -- reset ( Double Fault).

Following kernel changes

It is surprising how well all the parts of Linux operate together. I grabbed the kernel source and the most recent patch. I just unpacked it and clicked and kompare brought it up ( gotta love that file 'magic' ) and I could browse through the changes and see what was affected. If you understand the overall structure, it is easy to see what these things do. I am confused though, if this patch '2.6.29.2' and version is 2 patch level is 6 and sub-level is 29, where does extra version .2 fit in that picture and what does it mean? Of course I am being facetious. However, I really am wondering what a 'Temporary Tasmanian Devil' does.

I may do something odd and make a patch so I can take the kernel versions as they are modded and just patch them to my liking as a Kos style implementation.

Link to Linux Commando, command for console and it is ALT and period together to recall the operand from previous commands in order backward with each new press. Nice feature and saves some ambiguity.

Comparing kernel foo

This is some old 32 bit code from Kos in 'Intel ASM' register format and form.

BuildPm32Task: ; Create Physical Pointer To this TSS len 256 Mov Edi,PhysicalBaseTss ; TSS Mov Eax,WhichFault ; Mov Ecx,SizeOfTss ; Mul Ecx ; Index In Tss segment to this TSS add Eax,Edi ; And the Base Mov Ebx,Eax ; Save for Fill function Mov Dx,8900h ; Type and Granularity of size Mov Ebp,WhichFault ; Shl Ebp,3 ; *8 for Idt Call BuildTssInGdt ; Set it in the GDT to Tss link Mov Dx,8500h ; Type and Size Call PutGdtTsInIdt ; Set the link Idt to Gdt to Tss Mov Edi,Ebx ; This TSS Push Edi Call InstallTssDataP32 Call PutPortAccessV86 Pop Edi Ret

This is some code from the Linux kernel which is comparable.

/*H:610 Like the IDT, we never simply use the GDT the Guest gives us. We keep * a GDT for each CPU, and copy across the Guest's entries each time we want to * run the Guest on that CPU. * * This routine is called at boot or modprobe time for each CPU to set up the * constant GDT entries: the ones which are the same no matter what Guest we're * running. */ void setup_default_gdt_entries(struct lguest_ro_state *state) { struct desc_struct *gdt = state->guest_gdt; unsigned long tss = (unsigned long)&state->guest_tss; /* The Switcher segments are full 0-4G segments, privilege level 0 */ gdt[GDT_ENTRY_LGUEST_CS] = FULL_EXEC_SEGMENT; gdt[GDT_ENTRY_LGUEST_DS] = FULL_SEGMENT; /* The TSS segment refers to the TSS entry for this particular CPU. * Forgive the magic flags: the 0x8900 means the entry is Present, it's * privilege level 0 Available 386 TSS system segment, and the 0x67 * means Saturn is eclipsed by Mercury in the twelfth house. */ gdt[GDT_ENTRY_TSS].a = 0x00000067 | (tss << 16); gdt[GDT_ENTRY_TSS].b = 0x00008900 | (tss & 0xFF000000) | ((tss >> 16) & 0x000000FF); }

I am sure this just looks like caca to most everybody else, but I wrote original code in the BIOS for 386 protected mode switch and this shtuff is just second nature to me. When I was young I used to put those odd comments in the code and as time went by I quit doing it because it was funny the first time I reviewed the code and then just got irritating and I removed them all.

As much as I would rather use C , the advantage for a specific processor is nil in my opinion. I often rewrote these core code modules for different processors and I really like RISC big-endian machine code better, but Intel was the most common place to get work. I also use the 'Flat' real mode to play with VM before I switched. I also incorporated a full debugger that was invoked on a fault. That was my first design. It did disassembly and was rock solid before I did any more of the design.

It might scare somebody to drop to a debugger, but it was always my opinion that if any flaw existed that would stop the machine or fail in some bad way, I wanted to be all over that and know why. As the code is designed, I have not had a flaw in two decades now. I suppose it is possible, however I spent hours examining all the critical aspects of task switches, memory, and even timed every instruction of every module.

The silly thing is that I am the only person who ever used this OS.

The extensions to the OS are all written in pseudo code and all higher level functions beyond the main operating loop and memory/task management are purely gospel symbolic code that is executed as single steps. That took me nearly a year to design.

Something strange that I ran into the other day tells me that if I don't look deeply and have full knowledge of the source, it can be twisted, even if it is intended to be for a good purpose, back doors can be a major risk. Specifically I am referring to a self concealing compiling source sequence obfuscation. If you understand the process of bootstrap from the first hexadecimal code, you know who I'm talking about.

Linux fault handling

This is a side trip into the unknown. So I got this bug that I wanted to do fault handling inside my program. I learned a lot of things and the differences between Linux/Unix POSIX and Windows. It is a much better way to handle things, but very difficult to manage for most. Here is a link from Linux Journal ( somewhat dated 2003), but it has a good framework. I had to make some changes, but I managed to get the job done.

They assume certain things about architecture that are not a given and I found a missing include, but these things are not show stoppers for a programmer. A person wouldn't be interested in the code unless they knew C and POSIX and could run gcc and understand "-fexceptions" and stack frames and library naming and many other things.

Great fun and it is rewarding in the fact that I have a new feature for antfarmgl. I always never make mistakes so it will never get used, but it is there for others who might modify code and generate exceptions.

I already knew about try/catch in C++,java,Python etal, but I wanted to look directly into the face of the sun for my own understanding of architecture.

You should be warned that a knowledge of one of the dark arts may be required to understand this and if you are not familiar with the dark side assembly ways you might want to pass on the link.

Contributors

Automated Intelligence

Automated Intelligence
Auftrag der unendlichen LOL katzen