nirendra's blog

Communicating with kernel

Sun, 07 Jan 2007 20:24:32 -0800

According to Wikipedia, an operating system usually segregates virtual memory into kernel space and user space. Kernel space is strictly reserved for running the kernel, device drivers and kernel extensions. In most operating systems, kernel memory is never swapped out to disk. User space is the memory area where all user mode applications work and this memory can be swapped out when necessary.

A user application cannot access kernel space directly and similarly kernel code cannot access the user space without checking whether the page is present in memory or swapped out. Even though these can not access each other directly, user and kernel space can communicate with each other using variety of ways. This is an effort to summarize those used under Linux:

System calls

As explained in manual of syscall(2), The system call is the fundamental interface between an application and the kernel.

A system call is a request by a running task to the kernel to provide some sort of service on its behalf. In general, the kernel services invoked by system calls comprise an abstraction layer between hardware and user-space programs.

The way system calls are handled is up to the processor. Usually, a call to the kernel is due to an interrupt or exception; in the call, there is a request to execute something special.

Actual code for the system_call entry point can be found in /usr/src/linux/kernel/sys_call.S and the code for many of the system calls can be found in /usr/src/linux/kernel/sys.c. Code for the rest is distributed throughout the source files. Some system calls, like fork, have their own source file (e.g., kernel/fork.c).

Pros: Well documented, easy to use interface.

ioctl

Even though ioctl is one of the system calls, its usefulness tempts me to talk about it separately. Sometimes referred to as Swiss army knife, ioctl stands for input/output control and is used to manipulate a character device via a file descriptor. ioctl is a catch-all function that takes a device, a request and a variable number of other parameters. The devices, requests and other parameters are then defined in header files.

Pros: Provides hardware level access to programmers.
Cons: device and requests are poorly documented and platform specific.

procfs

The /proc file system (procfs) is a special file system in the Linux kernel. It is a virtual file system, i.e. not associated with a block device but exists only in memory.

The files in the procfs are there to allow userland programs access to certain information from the kernel (like process information in /proc/[0-9]+/), but also for debug purposes (like /proc/ksyms).

Pros: Uniform interface to retrieve process information from command line.
Cons: Difficult to figure out required information as too much miscellaneous information exposed through single file system.

Sysfs

Sysfs is a virtual file system provided by the 2.6 Linux kernel. Sysfs exports information about devices and drivers from the kernel device model to userspace, and is also used for configuration.

Sysfs is designed to export the information present in the device tree which would then no longer clutter up procfs.

For each object added in the driver model tree (drivers, devices including class devices) a directory in sysfs is created.

The parent/child relationship is reflected with subdirectories under /sys/devices/ (reflecting the physical layout). The subdirectory /sys/bus/ is populated with symbolic links, reflecting how the devices belong to different busses. /sys/class/ shows devices grouped according to classes, like network, while /sys/block/ contains the block devices.

Pros: Uniform interface to retrieve/update device specific information

Netlink sockets

Netlink socket is a special IPC used for transferring information between kernel and user-space processes. It provides a full-duplex communication link between the two by way of standard socket APIs for user-space processes and a special kernel API for kernel modules.

Netlink socket uses the address family AF_NETLINK.

The standard socket APIs-socket(), sendmsg(), recvmsg() and close()-can be used by user-space applications to access netlink socket.

Pros: It is a nontrivial task to add system calls, ioctls or proc files for new features; with the risk of polluting the kernel and damaging the stability of the system. Netlink socket is simple, though: only a constant, the protocol type, needs to be added to netlink.h.

relay

Relayfs is, yet another virtual filesystem implemented by the kernel; it must be explicitly mounted by user space to be available. Kernel code can then create a relay with relay_open(); it will show up as a file under relayfs.

User space can then open the relay and employ all of the usual file operations - including mmap() and poll() - to exchange data with the kernel.

To an application, a relayfs file descriptor looks much like a Unix-domain socket, except that the other end is a piece of kernel code rather than another process.

The interface on the kernel side is a bit more complex. The expected relay_read() and relay_write() functions exist and can be used to move data to and from user space.

But relayfs also exposes much of the internal structure to kernel code that needs to know about it. So special-purpose code can obtain a pointer into the relayfs buffer and copy data there directly.

Pros: Easier to pass large amount of information to and from kernel space.

debugfs

debugfs is a in-kernel filesystem just for putting debugging stuff some place other than proc and sysfs, and which is easier than both of them to use.

debugfs is meant for putting stuff that kernel developers need to see exported to userspace, yet don't always want hanging around.

To create a file using debugfs the call is just:
struct dentry *debugfs_create_file(const char *name, mode_t mode, struct dentry *parent, void *data, struct file_operations *fops);

To export a single value to userspace:
struct dentry *debugfs_create_u8(const char *name, mode_t mode, struct dentry *parent, u8 *value);

That's it, one line of code and a variable can be read and written to from userspace.

Pros: Shortens development time with debugging information easily available.

Firmware loading

While most computer peripherals work right "out of the box," some will not function properly until the host system has downloaded a blob of binary firmware.

The end result is that the recommended way of dealing with devices needing firmware downloads is to have a user-space process handle it.

In the new scheme, a device driver needing firmware for a particular device makes a call to:
int request_firmware(struct firmware **fw, const char *name, struct device *device);

Here, name is the name of the relevant device, and device is its device model entry. This call will create a directory with the given name under /sys/class/firmware and populate it with two files called loading and data. A hotplug event is then generated which, presumably, will inspire user space to find some firmware to feed the device.

The resulting user-space process starts by setting the loading sysfs attribute to a value of one. The actual firmware can then be written to the data file; when the process is complete, the loading file should be set back to zero. At that point, request_firmware() will return to the driver with fw pointing to the actual firmware data. The user-space process can chose to abort the firmware load by writing -1 to the loading attribute.

When the driver has loaded the firmware into its device, it should free up the associated memory with:
void release_firmware(struct firmware *fw);

Though, this is specific to firmware loading, but is sometimes used for passing large amount of data to the kernel space.

Kernel mode HTML parser

Sun, 23 Jul 2006 03:04:01 -0700

System calls are means through which user level processes can communicate with kernel. Though Linux kernel allows kernel code to invoke system calls.

This is not generally considered a good idea in terms of debugging, maintaining and porting the code. But if performance or size are absolutely necessary porting applications on kernel seems to have huge benefits.

The gain of performance comes for costly user/kernel space transition and associated data passing.

In order to measure timing benefits I implemented a rudimentary HTML parser in kernel space and a similar parser in userland.

Code snippets from kernel module for reading a html file and removing the html tags is as follows(Complete source available here):

	best = ~0;
	measure_time(0);
	tsc = best;
	printk(KERN_INFO "Time taken for no code: %ld\n", tsc);

	/*Measure time of reading a file*/
	/*Prepare to invoke system call*/
	fs = get_fs();	/*Save previous value*/
     	set_fs(get_ds());	/*use kernel limit*/
	/*Call system call*/
	fd = filp_open(FILE_NAME, O_RDONLY, 0600);

	if(fd->f_op && fd->f_op->read){
	    best = ~0;
	    measure_time(fd->f_op->read(fd, html, 1000, &fd->f_pos));
	    printk(KERN_INFO "Time taken by read: %ld\n", best-tsc);
	    parse_html(html, text); /*Parse html to text*/
	    printk(KERN_INFO "Parsed text: %s", text);
	}

This code parses the HTML by calling an ugly parser parse_html(Defined in common.h available here) which strips out the html tags.

While part of similar userland code is as follows(Complete source available here):

	/*time rdsc, i.e. no code*/
	best =~ 0;
	measure_time(0);
	tsc = best;
	printf("Time taken for no code: %ld\n", tsc);
	
	/*Measure time for reading a file*/
	fd = open(FILE_NAME, O_RDONLY, 0600);
	if(!fd){
	    printf("Error opening file\n");
	    exit(1);
	}
	best = ~0;
	measure_time(read(fd, html, 1000));
	printf("Time taken by read: %li\n", best - tsc);
	parse_html(html, text);
	printf("Parsed text: %s\n", text);

I collected following read time for first 5 runs:

Clock ticks/run	1st Run	2nd Run	3rd Run	4th Run	5th Run
Kernel HTML Parser	246	366	245	351	246
Userland HTML Parser	675	683	561	683	675

Thus, file read time in kernel outperforms userland code by around 3 times.

There are couple of interesting possibilities on porting application requiring high performance to kernel space. There already exists few including a Kernel mode web server. Ofcourse, the crash for a not properly tested module could cost more than their userland counterparts.

ReiserFS: To be or not to be

Mon, 17 Jul 2006 10:16:50 -0700

There seem to be quite alot of debate going on LKML over whether to include ReiserFS in kernel or not.
ReiserFS has been into problems over couple of things. Firstly, the way it was pushed by Hans Reiser was not liked by many.

Then there were talks over reliability of file system. As someone pointed out:
"The fact that reiserfs uses a single B-tree to store all of its data means that very entertaining things can happen if you lose a sector containing a high-level node in the tree.It's even more entertaining if you have image files (like initrd files) in reiserfs format stored in reiserfs, and you run the recovery program on the filesystem."

Another problem with ReiserFS is it's quest to integrate everything within filesystem. As an example it has plugins that can alter the symantics of files, like making files into directories inside which you could see meta-files like file/uid and file/size which contained meta-data and such accessible as normal files to all the unix tools. You could get things like chmod by just doing
'echo root >file/owner'.

Whether this is a good idea is quite debatable, as it is being long believed in Unix world that do one thing well and keep it simple. Next step in this direction could to parse the zip archives in kernel space for doing a 'cd linux-2.6.17.tar.bz2'(or is it already implemented) which does not sound like a good idea.
Moreover, this may require couple of changes in VFS.

I recently noticed missing readv system call in ReiserFS while calling it from kernel space.

Someone wrote an article on why ReiserFS is not included in kernel:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn

Although there seem to have been quite alot of development going on ReiserFS and it is installed as part of SUSE distributions. It seems some of the ideas implemented by this file system are unique and may be useful to other filesystems implemented in the future.

Java binary on Linux

Tue, 11 Jul 2006 09:26:07 -0700

Recently noticed support for Java binary on Linux kernel which means you can execute your Java applications simply as:
$ ./HelloWorld.class
And this can be achieved in following few steps(Assuming JDK is already installed and CLASSPATH properly configured):

Recompile your kernel with CONFIG_BINFMT_MISC option. This can be achieved as follows:
#cd /usr/src/linux; make menuconfig
Select "Executable file formats / Emulations" -> Kernel Support for MISC binaries
Save and exit. Follow /usr/src/linux/README for further information on compiling the kernel.
BINFMT_MISC can also be compiled as a independent module and inserted manually. This feature allows you to invoke almost any binary by simply typing it's name in shell. Refer to /usr/src/linux/Documentation/binfmt_misc.txt for more information on this.

Mount binfmt_misc and setup for Java executable:
# mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
# echo ':Java:M::\xca\xfe\xba\xbe::/usr/local/bin/javawrapper:' > /proc/sys/fs/binfmt_misc/register

Now execute the following:
# cat /usr/src/linux/Documentation/java.txt |grep -m3 -A 195 "Cut here"
and copy the first script as /usr/local/bin/javawrapper. This script will add the class file to classpath.
Compile the second C program as follow:
# gcc -O2 -o javaclassname javaclassname.c
# cp javaclassname /usr/local/bin
This executable is required to find the fully qualified class name i.e. for class Test.class in package foo.bar, it will return foo.bar.Test

5. And now the fun part. Just chmod any Java class to execute it.
# javac HelloWorld.java
# chmod +x HelloWorld.class
# ./HelloWorld.class

I gathered this from /usr/src/linux/Documentation/java.txt which can be referred for more information.

ISPs in India

Wed, 17 May 2006 11:06:04 -0700

An Internet service provider (ISP, also called Internet access provider or IAP) is a business or organization that offers users access to the Internet and related services. Many but not all ISPs are telephone companies.

Internet access reached India in the early 90’s. ERNet, a division of Department of Electronics (DoE), and NICNet (Department of Statistics) were the pioneers for providing this service. Both ERNet and NICNet were Government bodies, though had very different charters and growth histories.

The ERNet (Educational and Research Network) project was designed to provide Internet connectivity to the premier educational and research institutions of India, while NICNet was assigned the provision of Internet services primarily to Government departments and organizations. ERNet grew from a low-bandwidth, unreliable, shell1 and UUCP2 based Internet service to become the first to provide full TCP-IP3 access to dial-up modem4 customers through SLIP5 accounts around 1993.

NICNet began with shell-only access, at 2400 BPS, but started providing high speed TCP-IP access through 64 KBPS V-SAT links.

ERNet and NICNet are thus India’s first ISPs, though their operations have been shackled by the restrictions put upon them by Government regulations and policies of the Department of Telecom (DoT). Despite this, they were doing quite well in providing the essential Internet services to an Internet-starved India, until the advent of VSNL Internet services and the restrictive clampdown that followed.

Another provider of Internet services that preceded VSNL is the Software Technology Parks of India (STPI) Internet service. This service was permitted only for a restricted audience, the software exporters falling under the STP scheme of the DoE. STPI has been providing high-end Internet services through leased lines and dial-up links, in and around several of the Parks, including Bangalore, Hyderabad and NOIDA, through the respective SoftNET networks.

Entry of VSNL..

On 15th August 1995, VSNL launched the Gateway Internet Access Service, for providing public Internet access. Initially, DoT allowed VSNL the license to operate this service only in the 4 metros. By the VSNL charter, it is supposed to only provide international telecom gateways, not end-user services. Thus the name "Gateway Service" was used to cover up for direct service provision.

Starting with only dial-up shell and PPP8 access in the 4 metros, VSNL followed with leased-line access to subscribers, followed by the setting up of points of presence (POP) in Bangalore and Pune. The DoT has turned a blind eye to these license violations, and tacitly helped VSNL with post-facto ad hoc permissions along the way.

VSNL has, since the inception of GIAS, portrayed itself in the press as India’s only legitimate ISP, while forcing many restrictions on the other ISPs through DoT regulations and the telecom policy. The Telegraph Act of 1885, a pre-independence British law, has been repeatedly invoked by VSNL and interpreted to give itself extended powers while forcing the other ISPs to curtail their operations.

ISP monopoly never ceased to exist even after other players entering Indian market. There have been incidents of individuals facing problems regarding good quality internet connection.
Quoting an incident from Broadband blog:
"Well, to cut a long story short, this person works with a respectable computer company. He wanted to have a connection of his own to work. Since he lives in a place where there is no other service provider, he chose to trust his money in Tata’s brand name. However, despite repeated calls and efforts, he couldn’t trace the “executive” for his
connection. "

These and similar incidents might make one feel to go about opening their own ISPs.

Entanet, a supplier of business-to-business Internet services, has announced sometime back, that it is to offer a 'Virtual Pipe' option that takes advantage of a company's infrastructure. The Virtual Pipe enables ISPs, companies, societies, local authorities, schools, or any organization, to offer its own independently managed service.

Aimed at organizations that envisage connecting upwards of 50 users, customers of the service will have their own set of IP addresses and Radius server and bill their own customers directly, while Entanet manages all of the traffic and infrastructure behind the scenes.

Talking about Indian ISP is going to take time. Meanwhile, would like to hear your experiences about it.

Barcamp Pune

Fri, 05 May 2006 22:58:39 -0700

Foo Camp is an invite-only event for technology luminaries hosted in Sebastopol, CA at the O’Reilly headquarters. Though everyone is not invited in the event.
In response to criticism of Foo Camp, BarCamp was created as an open, welcoming, once-a-year event for geeks to camp out for a couple of days with wifi and smash their brains together.

It is an intense event with discussions, demos and interaction from attendees. Attendees must give a demo, a session, or help with one.

There is going to be a barcamp in Pune shortly. Visit this link to register for the event http://barcamp.org/BarCampPune

I will be there, will you be?

Mobber with online presence launched today

Fri, 24 Mar 2006 04:54:57 -0800

Presence is defined as knowing that a person is online, and on a connected device with a certain device profile. Each part of that definition is critical to fully understanding presence and how it is changing, and will continue to change, the face of business communications.

The most common use of presence today is the status indicator displayed on most instant messaging clients. A more simple everyday example is the 'on-hook' or 'off-hook' state of a telephone receiver, resulting in a distinctive ring tone for caller. Some states that offer extended information on the user's availiability are "free for chat", "away", "do not disturb", and "out to lunch", which are often seen on many modern instant messaging clients.

AOL Instant Messenger, Antepo, Skype, Microsoft Live Communication (LCS) Server, Lotus Instant Messaging and Web Conferencing, and other products provide facilities for presence management today. They differ in how they deliver this functionality and the audience for which this function is made available. But they are all similar in that presence is only conveyed to those on the same network.

However, in order to realize the full potential of presence it will have to be relayed in a consistent manner, regardless of the network that the individual may be connected to. Doing this is going to require new products based on standards, and efforts to make that happen are underway now.

There are several groups trying to standardize the presence protocol. XML based XMPP(Extensible Messaging and Presence Protocol) by Jabber, was approved in 2004 by the Internet Engineering Task Force (IETF) as an industry standard protocol, and is used in private enterprise instant messaging systems. Boeing selected XMPP to provide chat and presence services for the U.S. Army's Future Combat Systems (FCS) initiative.

SIP stands for Session Initiation Protocol, and is a signaling protocol used to establish sessions in an IP-based network. The SIP protocol is stewarded by the IETF and is being broadly extended and enhanced to support the requirements of secure instant messaging and presence management (SIMPLE) . Another leading standard (OASIS), also called the Liberty Alliance Project has released a specification for a presence management Web service within the context of the Liberty Alliance, a federated identity management solution.

Presence is a much hyped word in instant messaging domain. One of the startling factoid is that 40 percent of business IM use leads to a phone call. This points out that the core value of IM isn't messaging, but presence, that capability integral to IM systems that lets others know if you are online and available.

Will Web 2.0 change the definition of online presence? Noticed Mobber today. It shows you who's on a particular web page, allowing you to chat with them privately or in a group. It's their first day and they are already rocking.

For anyone who wants to chat with me or with anyone else looking at this page, I've added a Mobber bar to the bottom of the site - it's pretty slick. Talk to each other or ping me.

Entrepreneurs in India

Sun, 19 Mar 2006 20:01:24 -0800

Who is an Entrepreneur
Wikipedia introduces Entrepreneur as a loanword from the French language that refers to a person who undertakes and operates a new venture, and assumes some accountability for the inherent risks. Being in business or being an entrepreneur is about taking risks and confronting challenges

Entrepreneurs build companies that are specifically crafted to exploit a particular opportunity. This gives them an advantage over older companies that were designed in response to challenges of the past and must change to adapt to today’s requirements. Entrepreneurs can build new companies. They can also rejuvenate existing companies via buyouts and turnarounds. They can also build new companies inside existing companies, which can be called corporate entrepreneurship.

The will to spot opportunities and take risks in order to realize them is part of a person’s overall makeup, which is partly innate and partly a product of his upbringing. The best way to learn how to be an entrepreneur is to work at the side of a successful one. The problem is that entrepreneurs are understandably reluctant to hire those who cannot help them immediately. It appears that the best way to learn this is to work for a startup which offers more opportunities to learn Entrepreneural skills.

Risk-taking and opportunism go along with frugality. Really good entrepreneurs squeeze as much as possible out of limited amounts of cash. They leverage the money of others, and never invent the wheel when a good, cheap one is available in the marketplace. By keeping the rate at which they burn cash low, entrepreneurs can try a lot of ideas, most of which do not work, without losing because they ran out of money before they hit upon a workable value proposition.

Many "high-profile" entrepreneurial ventures seek venture capital or angel funding in order to raise capital to build the business. Many kinds of organizations now exist to support would-be entrepreneurs, including specialized government agencies, business incubators, science parks, and some NGOs.

Venture capital investments generally are high risk investments but offer the potential for above average returns. An angel investor (business angel in the UK, or simply angel) is an affluent individual who provides capital for a business start-up, usually in exchange for ownership equity. Unlike venture capitalists, angels typically do not manage the pooled money of others in a professionally-managed fund. However, angel investors often organize themselves into angel networks or angel groups to share research and pool their own investment capital.

in India...
According to Global Enterpreneurship Monitor(GEM) project India Report 2001, Due to social rigidities, Indian women are half as likely as men to be entrepreneurs. Younger, moderately educated, and reasonably well-off people are more likely to be entrepreneurs. The types of startups encountered (established within the last 42 months) were mostly consumer-oriented, comprising of trading activities; most have just about a handful of employees. Main sources of funds in decreasing order were personal, financial institutions, close family members, and government programs.
The wheels of India's bureaucracy still turn too slow for entrepreneurs, the educational system is not good at promoting entrepreneurial skills and attitudes, Indian institutes have not been as good as multinationals in R&D transfer, and India's physical infrastructure ranks lowest among the countries surveyed in the report - all prime areas for study and improvement by policymakers, academics and business leaders.

According to Vivek paul, former CEO Wipro "The stuff that’s been done in India is staggering in terms of range and depth. I don’t think that anyone can say that the work we’re doing is trivial. But the work we’re doing is under somebody else’s direction. Let me put it this way: For an engineer, there’s a big difference between discovering something, versus discovering something that you know somebody else says can be done. That difference is the difference between the service business and the products business. In the service business, what you’re doing is great stuff, but it is in some sense something that someone else told you to do. "
He further argues "If you look at that service business as leading to innovation and product outcomes, the answer is absolutely not. Frankly, I feel that when people work in a service business like ours, it’s almost like we give them a lobotomy. I don’t think - and I hope I’m wrong - you will see a single successful product startup coming out of people who were working at Wipro or any other similar companies. You’ll find that innovation comes from people who worked for Intel India; they’ll go off and come up with a new chip. Or someone at Cisco India will come up with a new router. Why that is, God knows. But I truly believe that there is some sort of inadvertent lobotomy that we give people. "

Is India a right place to start a startup? Following are few interesting observations:
1. People: India is a land of technologists. It is the best place for techies with similar interests to combine their skills and innovate than in any other countries.
2. Funding: Indian startups can survive for much longer time than in any other countries looking at the limited amount of cash requirement for running business here.

Does India lacks Entrepreneurship skills? India Venture Challenge organized by The Draper Fisher Jurvetson (DFJ) and TiE (The Indus Entrepreneurs) met with a stupendous response and saw participation of 125 budding and aspiring entrepreneurial teams from across India, representing both new ventures and existing early stage businesses.
According to Tim Draper, Founder and Managing Director of DFJ,“Entrepreneurial talent abounds in India and needs the right atmosphere and encouragement for the skills to be honed and met with success."

I believe it is required to create right environment to create successful business builders in India. To do this India should be focusing on following areas:
1. Create the right environment for success: Entrepreneurs should find it easy to start a business. To do so, most Indians would start slow with capital borrowed from family and friends, the CEO playing the role of salesman, a professional team assembled months or perhaps years after the business was created, and few, if any, external partners. Compare this with a start-up in the Silicon Valley: a Venture Capitalist (VC) or angel investor would be brought in early on; a professional management team would drive the business; a multi functional team would be assembled quickly; and partnerships would be explored early on to scale up the business.
To a large measure, culture shapes this style. Silicon Valley is abuzz with ideas to build global businesses; deals are continually being negotiated, teams are pulled together and partners are identified. There is almost unlimited access to multiple VCs and angel investors. Critical support services abound, including professional managers, legal firms, venture capitalists, angel investors, and placement agencies. Combine this with excellent infrastructure – connectivity, communication, and office space – and getting started is easy.
2. Ensure that entrepreneurs have access to the right skills: A survey McKinsey & Company conducted in 2000 revealed that most Indian start-up businesses face two skill gaps: entrepreneurial (how to manage business risks, build a team, identify and get funding) and functional (product development know-how, marketing skills, etc.). In other countries, entrepreneurs either gain these skills by hiring managers or have access to “support systems” such as universities or other institutions that may nurture many regional businesses. In addition, business schools give young graduates the skills and knowledge required for business today.
3. Ensure that entrepreneurs have access to “smart” capital: For a long time, Indian entrepreneurs have had little access to capital. It is true that in the last few years, several Venture Funds have entered the Indian market. And, while the sector is still in its infancy in India, VCs are providing capital as well as critical knowledge and access to potential partners, suppliers, and clients across the globe. However India has only a few angel investors who support an idea in the early stages before VCs become involved.
4. Enable networking and exchange: Entrepreneurs learn from experience - theirs and that of others. Much of the success of Indians in Silicon Valley is attributed to the experience, sharing and support TIE members have extended to young entrepreneurs.

My experiments with iPod

Sun, 25 Sep 2005 14:21:55 -0700

What is this iPod
iPod is a brand of portable digital audio player designed by Apple computer. The first three generations of iPod use two ARM 7TDMI-derived CPUs running at 90 MHz, while later models have variable speed chips with a peak of 80 MHz to save battery life.

Is this an article on iPod
This article is intended for iPod users on Linux and also for Linux users on iPod.

Locating your iPod
Before getting started with using iPod on Linux, the first step would be to locate the iPod on Linux. After connecting the iPod with USB port, check the location of iPod:

# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: Apple Model: iPod Rev: 1.50
Type: Direct-Access ANSI SCSI revision: 02

scsi0 means that iPod is connected to /dev/sda, scsi1 means that iPod is connected to /dev/sdb and so on.

I am assuming that /dev/sda is used throughout this article.

Mounting and unmounting iPod
Now is the time to start exploring iPod. For this we need to mount the iPod on Linux:

# mkdir /mnt/myiPodSongs
# mount -t vfat /dev/sda2 /mnt/myiPodSongs

After mounting iPod, the content of iPod can be browsed as a normal file system.

Once done, iPod can be unmounted as follows:
#umount /dev/sda2

Copying and playing music
Ok, this was neat, but what about playing some music.
On Linux, gtkpod can be used to copy and organize your music into playlists .

After installing gtkpod, mount point of iPod can be specified from "Edit->Edit Preferences".

Once the mount point is added, gtkpod let's you create and modify playlists, add or remove songs and even export your thunderbird or evolution data to the iPod.

Podcasting
Podcasts are radio-style shows for iPod. Unlike streaming audio, which requires you to listen in real time, podcasting lets you control how and when you hear your favorite shows.

Some of the podcasting clients available on Linux are amaroK or BashPodder, a command line utility to download podcasts.

Installing Linux on iPod
Now that we know how to use iPod on Linux, it's time to use Linux on iPod.

But why bother to install Linux on an iPod? because it includes loads of new games, movie player, voice recorder and the freedom to write new applications.

iPodLinux can be used to install Linux on iPod. First, second and third generation of iPods are officially supported by iPodLinux.

After playing with installer a bit I was able to install Linux on my 60GB photo iPod.

The port uses uClinux, a Linux flavor designed for devices that lack a memory management unit (MMU). While the iPod has some MMU-type capabilities, they are not sufficient to support the Linux kernel.

Before installing Linux, it will be a good idea to take backup of iPod boot loader and OS, which can be restored if something goes wrong:

# dd if=/dev/sda of=ipod_boot_sector_backup count=1
# dd if=/dev/sda1 of=ipod_os_partition_backup

After the backup is done, next step would be to create a new partition on iPod to hold root file system for Linux.

Partition can be created using "fdisk /dev/sda", I created following partitions for a 60GB iPod.

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1 8001 0 Empty
/dev/sda2 * 6 7296 58564957+ b W95 FAT32
/dev/sda3 2 5 32130 83 Linux

Once partition is done to hold root filesystem, create the filesystem with an ext3 journal:

# mke2fs -j /dev/sda3

and adjust the maximal mount count between two filesystem checks to never:
# tune2fs -c 0 /dev/sda3

To install Linux kernel and boot loader, download either

Official Boot loader(for 1,2 and 3 generation iPod)
or
Boot loader for 4G photo iPod

Also, download the latest kernel from nightly build.

Extract the kernel and boot loader in a directory. Assuming that name of the kernel is: uclinux-2.4.24-ipod2.bin

Now extract the apple image from the boot loader and create a new image including Linux and Apple OS:

# ./make_fw -3 -o apple_os.bin -e 0 ipod_os_partition_backup
# ./make_fw -3 -o my_sw.bin -i apple_os.bin -l uclinux-2.4.24-ipod2.bin loader.bin

NOTE: -3 is required in make_fw only for 4G iPods.

Once this is done, we can copy the new image and Kernel modules into iPod:

# dd if=my_sw.bin of=/dev/sda1
# mkdir /mnt/ipod
# mount -t ext3 /dev/sda3 /mnt/ipod
# cp -r lib /mnt/ipod
# umount /mnt/ipod

For userland installation, I downloaded the iPodLinux compress FS and performed the following steps to install filesystem:

Mount iPod root partition

# mount -t ext3 /dev/sda3 /mnt/ipod

Extract the filesystem

# mkdir /mnt/linuxos
# mount -o loop \
floydzilla-2005-08-28-ipod+iboy+doom.img /mnt/linuxos
#cp /mnt/linuxos /mnt/ipod

and finally, unmount the iPod partition

#umount /mnt/ipod
#umount /mnt/linuxos
#eject /dev/sda

And we are done. But before you start playing Doom on iPod, it needs to be rebooted.

To reboot disconnect your iPod from the USB port. If the iPod did not automatically reboot hold down the menu and the play buttons for 3 seconds to reboot it.
After reboot keep the back button pressed to boot Linux, otherwise the default interface will load.

Programming your iPod
First thing required to compile hello world on iPod would be ofcourse a compiler.
But why a new compiler? because iPod have ARM processors and a cross compiler is required to compile programs for this architecture. Download the compiler for uclinux from here.

After the cross compiler is installed, next thing to do would be to write a hello iPod program.

Once done, compile the program:

# arm-elf-gcc -o hello hello.c -elf2flt

copy the program into iPod:

# mount -t vfat /dev/sda2 /mnt/myIpodSongs
# mkdir /mnt/myIpodSongs/myprogs
# cp hello /mnt/myIpod/myprogs

and reboot the iPod(After unmounting the partition)

# cd $HOME
# umount /mnt/myIpodSongs
# eject /dev/sda

After booting into floydzilla, goto "File browser->/mnt/myprogs" and click on the "hello" to see output of hello world on iPod.

Click on read more to view some of the images.

Rainy Amsterdam

Sun, 15 Jan 2006 18:09:14 -0800

Amsterdam is often referred to as one of the most colorful cities in the world. And rightly so where else do you find so many places of interest, famous museums, cozy pubs and flower markets.
Amsterdam is an unusual city in that it has all the advantages of a big city – culture, history, food, entertainment, good transport – with relatively few of the disadvantages: it is physically small, beautiful, relatively quiet, and largely thanks to the canals, has relatively little traffic.

I reached Amsterdam last week. Netherland is part of Schengen countries so it is possible to visit most of the Europe now.

Went to Belgium this weekend, the home town of Tintin. Here are some of the pics of my trip to Belgium this weekend.

Google on command line

Sun, 26 Jun 2005 07:52:33 -0700

In my quest for writing webservices in C, I wrote a small command line utility for collecting information from Google.

Released the first (0.1) version of Gcmd this weekend. Project home page is located at http://gcmd.sf.net. Being POSIX compliant as of now, it should work under variety of UNIX like Operating systems and Cygwin.

Following features are currently included:

Google command line search for a keyword.

Compare search results for two keywords in Google.

For a given URL collect the following statistics:
     i. Websites having link to this URL.
     ii. Websites similar to the given URL.
     iii.Website containing this URL in their page.

Spelling suggestion for a word or phrase.

and I am planning to add few more features.

So download this utility from here and let me know what you think about it.

My article in linuxgazette

Fri, 03 Jun 2005 10:51:11 -0700

My article on Proc file system has been published in Linux Gazette's June Issue.

Future of blogging

Wed, 17 May 2006 11:17:14 -0700

According to news.com, a new blog is created every 7.4 seconds. That adds up to 12,000 new blogs a day, 275,000 posts a day and 10,800 updates an hour. Technorati, a search engine that monitors blogs, tracked more than 8 million online diaries as of March 21, 2005, up from 100,000 just two years ago.

Blogging is really catching up. Indian government is also planning to open doors to Indian bloggers.

I was wondering on how future of blogging will look like.

Blog Weblog wrote that Autoblogging will be the wave of future. According to them, some examples of autoblogging could be :
1. autoblogging by email
2. autoblogging via mobile phone
3. autoblogging in a supermarket with a PDA

A new term Moblog is emerging in blogging these days.
Recent interest in wireless blogging seems to revolve around SMS. Essentially this is e-mail blogging from a cellular phone using an SMS to E-mail gateway.
Though, Blogging using PDA is also becoming common. People have started blogging using Camera phones, and there are products to even update a blog using phones.

People are finding Audio blogging as interesting. I recently noticed a site audioblogger which seems to be an extension of Blogger and provides unlimited audio posts. Another site audblog allows bloggers to post audio to their blogs using any phone.

Talking about expressing ideas, Video blogs seems to be the next extension to audio blogging. Vidblog contains a collection of video blogs, and the number seemed fairly surprising to me. Search engine giant Google is also looking towards video blogging, they have already started accepting videos.

There are so many blogs on web that it becomes difficult to keep track of them. RSS, ATOM, RDF, OPML (Outline Processor Markup Language) are few of the popular specifications of syndicating and sharing weblogs. OPML is a file format which can be used to share RSS subscriptions. It can be used in RSS aggregators like Bloglines or Akregator to import an OPML file to subscribe to all the feeds.

Lot of Companies are encouraging their employees to blog. Google was probably the first one to create a corporate blog, and now there are others including Microsoft, Macromedia, Yahoo search and Ask Jeeves.

With all these new changes taking place in blogging, it makes me think how it will look like in future. Adding few more points on what Fabrice wrote on this:

Blogging and social networking softwares will converge.

websites will become more blog like.

Corporate blogging will grow. Companies are already finding it as a way for the employees to express their thoughts and concerns.

There will be areas other than technology and politics emerging in blogs.

Based on blogger's interests, new blogging groups will start emerging.

New business models for blogging will emerge. Companies like google, yahoo, msn, bloglines are already working on providing blogging services, and it might take form of something similar to having a premium blogging account with more space and more features with some extra charges.

Harmony in Java community

Wed, 17 May 2006 11:14:56 -0700

Apache came up with a proposal to develop a open source implementation of Java called Harmony.

Though, Harmony FAQ shows that they are going to have tough time making sure that they are not voilating Sun's license.

Talking about license, there have been lot of discussions on Sun's license earlier also. There were efforts to develop a free Java implementation including GNU classpath and Kaffe but both of these were licensed under GPL.

The special thing about Harmony is that it is going to be released under Apache's license.

Though initiative of developing a free VM raises few thoughts:

Will it always be released at the same time as a new release of Java by Sun?
I think not for now atleast. As it appears that development of Java VM is not a small task and to catch up with current state Java VM should take quite alot of time.

Can it be better than Java?
No, as a mail archieve indicates, it has to confirm to J2SE specification or it can not be called "Java".

What will be the future of Java if this really "catches" up?
There will be lot of diverse implementation of Java "like" platforms, each taking up code base of Harmony and implementing their own VM.

Surely, this puts Sun into a interesting position.
Sun has been trying to improve relationship with Open source people, inviting others to develop specification for their next java release, or contribute to it.

I personally feel development of Harmony will create more diversification of Java community.

Google suggest shows incorrect result

Sun, 26 Jun 2005 07:51:39 -0700

I was impressed by the intersting user interface of Google suggest. It uses XMLHttpRequest for making callback to Google and updates the search results on every key press.

The JavaScript code within it, calls a routine callGoogle, which makes call in the following format (On typing goog in English locale): http://www.google.com/complete/search?hl=en&js=true&qu=goog. The URL returns two arrays, one with possible results and another with number of results.

This interface was developed by one of the Google’s employee in his 20% time, which is a program where Google allows their employees to devote 20% of their working hours to any project they choose.

But does this really shows the right results.

Screenshot below shows 155,000,000 results for keyword "google" in Google complete and 281,000,000 results when searching the keyword in google.com itself.