Nirendra Awasthi - Observations

Kernel mode HTML parser

Sun, 23 Jul 2006 03:04:01 -0700

System calls are means through which user level processes can communicate with kernel. Though Linux kernel allows kernel code to invoke system calls.

This is not generally considered a good idea in terms of debugging, maintaining and porting the code. But if performance or size are absolutely necessary porting applications on kernel seems to have huge benefits.

The gain of performance comes for costly user/kernel space transition and associated data passing.

In order to measure timing benefits I implemented a rudimentary HTML parser in kernel space and a similar parser in userland.

Code snippets from kernel module for reading a html file and removing the html tags is as follows(Complete source available here):

	best = ~0;
	measure_time(0);
	tsc = best;
	printk(KERN_INFO "Time taken for no code: %ld\n", tsc);

	/*Measure time of reading a file*/
	/*Prepare to invoke system call*/
	fs = get_fs();	/*Save previous value*/
     	set_fs(get_ds());	/*use kernel limit*/
	/*Call system call*/
	fd = filp_open(FILE_NAME, O_RDONLY, 0600);

	if(fd->f_op && fd->f_op->read){
	    best = ~0;
	    measure_time(fd->f_op->read(fd, html, 1000, &fd->f_pos));
	    printk(KERN_INFO "Time taken by read: %ld\n", best-tsc);
	    parse_html(html, text); /*Parse html to text*/
	    printk(KERN_INFO "Parsed text: %s", text);
	}

This code parses the HTML by calling an ugly parser parse_html(Defined in common.h available here) which strips out the html tags.

While part of similar userland code is as follows(Complete source available here):

	/*time rdsc, i.e. no code*/
	best =~ 0;
	measure_time(0);
	tsc = best;
	printf("Time taken for no code: %ld\n", tsc);
	
	/*Measure time for reading a file*/
	fd = open(FILE_NAME, O_RDONLY, 0600);
	if(!fd){
	    printf("Error opening file\n");
	    exit(1);
	}
	best = ~0;
	measure_time(read(fd, html, 1000));
	printf("Time taken by read: %li\n", best - tsc);
	parse_html(html, text);
	printf("Parsed text: %s\n", text);

I collected following read time for first 5 runs:

Clock ticks/run	1st Run	2nd Run	3rd Run	4th Run	5th Run
Kernel HTML Parser	246	366	245	351	246
Userland HTML Parser	675	683	561	683	675

Thus, file read time in kernel outperforms userland code by around 3 times.

There are couple of interesting possibilities on porting application requiring high performance to kernel space. There already exists few including a Kernel mode web server. Ofcourse, the crash for a not properly tested module could cost more than their userland counterparts.

ReiserFS: To be or not to be

Mon, 17 Jul 2006 10:16:50 -0700

There seem to be quite alot of debate going on LKML over whether to include ReiserFS in kernel or not.
ReiserFS has been into problems over couple of things. Firstly, the way it was pushed by Hans Reiser was not liked by many.

Then there were talks over reliability of file system. As someone pointed out:
"The fact that reiserfs uses a single B-tree to store all of its data means that very entertaining things can happen if you lose a sector containing a high-level node in the tree.It's even more entertaining if you have image files (like initrd files) in reiserfs format stored in reiserfs, and you run the recovery program on the filesystem."

Another problem with ReiserFS is it's quest to integrate everything within filesystem. As an example it has plugins that can alter the symantics of files, like making files into directories inside which you could see meta-files like file/uid and file/size which contained meta-data and such accessible as normal files to all the unix tools. You could get things like chmod by just doing
'echo root >file/owner'.

Whether this is a good idea is quite debatable, as it is being long believed in Unix world that do one thing well and keep it simple. Next step in this direction could to parse the zip archives in kernel space for doing a 'cd linux-2.6.17.tar.bz2'(or is it already implemented) which does not sound like a good idea.
Moreover, this may require couple of changes in VFS.

I recently noticed missing readv system call in ReiserFS while calling it from kernel space.

Someone wrote an article on why ReiserFS is not included in kernel:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn

Although there seem to have been quite alot of development going on ReiserFS and it is installed as part of SUSE distributions. It seems some of the ideas implemented by this file system are unique and may be useful to other filesystems implemented in the future.

Java binary on Linux

Tue, 11 Jul 2006 09:26:07 -0700

Recently noticed support for Java binary on Linux kernel which means you can execute your Java applications simply as:
$ ./HelloWorld.class
And this can be achieved in following few steps(Assuming JDK is already installed and CLASSPATH properly configured):

Recompile your kernel with CONFIG_BINFMT_MISC option. This can be achieved as follows:
#cd /usr/src/linux; make menuconfig
Select "Executable file formats / Emulations" -> Kernel Support for MISC binaries
Save and exit. Follow /usr/src/linux/README for further information on compiling the kernel.
BINFMT_MISC can also be compiled as a independent module and inserted manually. This feature allows you to invoke almost any binary by simply typing it's name in shell. Refer to /usr/src/linux/Documentation/binfmt_misc.txt for more information on this.

Mount binfmt_misc and setup for Java executable:
# mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
# echo ':Java:M::\xca\xfe\xba\xbe::/usr/local/bin/javawrapper:' > /proc/sys/fs/binfmt_misc/register

Now execute the following:
# cat /usr/src/linux/Documentation/java.txt |grep -m3 -A 195 "Cut here"
and copy the first script as /usr/local/bin/javawrapper. This script will add the class file to classpath.
Compile the second C program as follow:
# gcc -O2 -o javaclassname javaclassname.c
# cp javaclassname /usr/local/bin
This executable is required to find the fully qualified class name i.e. for class Test.class in package foo.bar, it will return foo.bar.Test

5. And now the fun part. Just chmod any Java class to execute it.
# javac HelloWorld.java
# chmod +x HelloWorld.class
# ./HelloWorld.class

I gathered this from /usr/src/linux/Documentation/java.txt which can be referred for more information.

ISPs in India

Wed, 17 May 2006 11:06:04 -0700

An Internet service provider (ISP, also called Internet access provider or IAP) is a business or organization that offers users access to the Internet and related services. Many but not all ISPs are telephone companies.

Internet access reached India in the early 90’s. ERNet, a division of Department of Electronics (DoE), and NICNet (Department of Statistics) were the pioneers for providing this service. Both ERNet and NICNet were Government bodies, though had very different charters and growth histories.

The ERNet (Educational and Research Network) project was designed to provide Internet connectivity to the premier educational and research institutions of India, while NICNet was assigned the provision of Internet services primarily to Government departments and organizations. ERNet grew from a low-bandwidth, unreliable, shell1 and UUCP2 based Internet service to become the first to provide full TCP-IP3 access to dial-up modem4 customers through SLIP5 accounts around 1993.

NICNet began with shell-only access, at 2400 BPS, but started providing high speed TCP-IP access through 64 KBPS V-SAT links.

ERNet and NICNet are thus India’s first ISPs, though their operations have been shackled by the restrictions put upon them by Government regulations and policies of the Department of Telecom (DoT). Despite this, they were doing quite well in providing the essential Internet services to an Internet-starved India, until the advent of VSNL Internet services and the restrictive clampdown that followed.

Another provider of Internet services that preceded VSNL is the Software Technology Parks of India (STPI) Internet service. This service was permitted only for a restricted audience, the software exporters falling under the STP scheme of the DoE. STPI has been providing high-end Internet services through leased lines and dial-up links, in and around several of the Parks, including Bangalore, Hyderabad and NOIDA, through the respective SoftNET networks.

Entry of VSNL..

On 15th August 1995, VSNL launched the Gateway Internet Access Service, for providing public Internet access. Initially, DoT allowed VSNL the license to operate this service only in the 4 metros. By the VSNL charter, it is supposed to only provide international telecom gateways, not end-user services. Thus the name "Gateway Service" was used to cover up for direct service provision.

Starting with only dial-up shell and PPP8 access in the 4 metros, VSNL followed with leased-line access to subscribers, followed by the setting up of points of presence (POP) in Bangalore and Pune. The DoT has turned a blind eye to these license violations, and tacitly helped VSNL with post-facto ad hoc permissions along the way.

VSNL has, since the inception of GIAS, portrayed itself in the press as India’s only legitimate ISP, while forcing many restrictions on the other ISPs through DoT regulations and the telecom policy. The Telegraph Act of 1885, a pre-independence British law, has been repeatedly invoked by VSNL and interpreted to give itself extended powers while forcing the other ISPs to curtail their operations.

ISP monopoly never ceased to exist even after other players entering Indian market. There have been incidents of individuals facing problems regarding good quality internet connection.
Quoting an incident from Broadband blog:
"Well, to cut a long story short, this person works with a respectable computer company. He wanted to have a connection of his own to work. Since he lives in a place where there is no other service provider, he chose to trust his money in Tata’s brand name. However, despite repeated calls and efforts, he couldn’t trace the “executive” for his
connection. "

These and similar incidents might make one feel to go about opening their own ISPs.

Entanet, a supplier of business-to-business Internet services, has announced sometime back, that it is to offer a 'Virtual Pipe' option that takes advantage of a company's infrastructure. The Virtual Pipe enables ISPs, companies, societies, local authorities, schools, or any organization, to offer its own independently managed service.

Aimed at organizations that envisage connecting upwards of 50 users, customers of the service will have their own set of IP addresses and Radius server and bill their own customers directly, while Entanet manages all of the traffic and infrastructure behind the scenes.

Talking about Indian ISP is going to take time. Meanwhile, would like to hear your experiences about it.

Entrepreneurs in India

Sun, 19 Mar 2006 20:01:24 -0800

Who is an Entrepreneur
Wikipedia introduces Entrepreneur as a loanword from the French language that refers to a person who undertakes and operates a new venture, and assumes some accountability for the inherent risks. Being in business or being an entrepreneur is about taking risks and confronting challenges

Entrepreneurs build companies that are specifically crafted to exploit a particular opportunity. This gives them an advantage over older companies that were designed in response to challenges of the past and must change to adapt to today’s requirements. Entrepreneurs can build new companies. They can also rejuvenate existing companies via buyouts and turnarounds. They can also build new companies inside existing companies, which can be called corporate entrepreneurship.

The will to spot opportunities and take risks in order to realize them is part of a person’s overall makeup, which is partly innate and partly a product of his upbringing. The best way to learn how to be an entrepreneur is to work at the side of a successful one. The problem is that entrepreneurs are understandably reluctant to hire those who cannot help them immediately. It appears that the best way to learn this is to work for a startup which offers more opportunities to learn Entrepreneural skills.

Risk-taking and opportunism go along with frugality. Really good entrepreneurs squeeze as much as possible out of limited amounts of cash. They leverage the money of others, and never invent the wheel when a good, cheap one is available in the marketplace. By keeping the rate at which they burn cash low, entrepreneurs can try a lot of ideas, most of which do not work, without losing because they ran out of money before they hit upon a workable value proposition.

Many "high-profile" entrepreneurial ventures seek venture capital or angel funding in order to raise capital to build the business. Many kinds of organizations now exist to support would-be entrepreneurs, including specialized government agencies, business incubators, science parks, and some NGOs.

Venture capital investments generally are high risk investments but offer the potential for above average returns. An angel investor (business angel in the UK, or simply angel) is an affluent individual who provides capital for a business start-up, usually in exchange for ownership equity. Unlike venture capitalists, angels typically do not manage the pooled money of others in a professionally-managed fund. However, angel investors often organize themselves into angel networks or angel groups to share research and pool their own investment capital.

in India...
According to Global Enterpreneurship Monitor(GEM) project India Report 2001, Due to social rigidities, Indian women are half as likely as men to be entrepreneurs. Younger, moderately educated, and reasonably well-off people are more likely to be entrepreneurs. The types of startups encountered (established within the last 42 months) were mostly consumer-oriented, comprising of trading activities; most have just about a handful of employees. Main sources of funds in decreasing order were personal, financial institutions, close family members, and government programs.
The wheels of India's bureaucracy still turn too slow for entrepreneurs, the educational system is not good at promoting entrepreneurial skills and attitudes, Indian institutes have not been as good as multinationals in R&D transfer, and India's physical infrastructure ranks lowest among the countries surveyed in the report - all prime areas for study and improvement by policymakers, academics and business leaders.

According to Vivek paul, former CEO Wipro "The stuff that’s been done in India is staggering in terms of range and depth. I don’t think that anyone can say that the work we’re doing is trivial. But the work we’re doing is under somebody else’s direction. Let me put it this way: For an engineer, there’s a big difference between discovering something, versus discovering something that you know somebody else says can be done. That difference is the difference between the service business and the products business. In the service business, what you’re doing is great stuff, but it is in some sense something that someone else told you to do. "
He further argues "If you look at that service business as leading to innovation and product outcomes, the answer is absolutely not. Frankly, I feel that when people work in a service business like ours, it’s almost like we give them a lobotomy. I don’t think - and I hope I’m wrong - you will see a single successful product startup coming out of people who were working at Wipro or any other similar companies. You’ll find that innovation comes from people who worked for Intel India; they’ll go off and come up with a new chip. Or someone at Cisco India will come up with a new router. Why that is, God knows. But I truly believe that there is some sort of inadvertent lobotomy that we give people. "

Is India a right place to start a startup? Following are few interesting observations:
1. People: India is a land of technologists. It is the best place for techies with similar interests to combine their skills and innovate than in any other countries.
2. Funding: Indian startups can survive for much longer time than in any other countries looking at the limited amount of cash requirement for running business here.

Does India lacks Entrepreneurship skills? India Venture Challenge organized by The Draper Fisher Jurvetson (DFJ) and TiE (The Indus Entrepreneurs) met with a stupendous response and saw participation of 125 budding and aspiring entrepreneurial teams from across India, representing both new ventures and existing early stage businesses.
According to Tim Draper, Founder and Managing Director of DFJ,“Entrepreneurial talent abounds in India and needs the right atmosphere and encouragement for the skills to be honed and met with success."

I believe it is required to create right environment to create successful business builders in India. To do this India should be focusing on following areas:
1. Create the right environment for success: Entrepreneurs should find it easy to start a business. To do so, most Indians would start slow with capital borrowed from family and friends, the CEO playing the role of salesman, a professional team assembled months or perhaps years after the business was created, and few, if any, external partners. Compare this with a start-up in the Silicon Valley: a Venture Capitalist (VC) or angel investor would be brought in early on; a professional management team would drive the business; a multi functional team would be assembled quickly; and partnerships would be explored early on to scale up the business.
To a large measure, culture shapes this style. Silicon Valley is abuzz with ideas to build global businesses; deals are continually being negotiated, teams are pulled together and partners are identified. There is almost unlimited access to multiple VCs and angel investors. Critical support services abound, including professional managers, legal firms, venture capitalists, angel investors, and placement agencies. Combine this with excellent infrastructure – connectivity, communication, and office space – and getting started is easy.
2. Ensure that entrepreneurs have access to the right skills: A survey McKinsey & Company conducted in 2000 revealed that most Indian start-up businesses face two skill gaps: entrepreneurial (how to manage business risks, build a team, identify and get funding) and functional (product development know-how, marketing skills, etc.). In other countries, entrepreneurs either gain these skills by hiring managers or have access to “support systems” such as universities or other institutions that may nurture many regional businesses. In addition, business schools give young graduates the skills and knowledge required for business today.
3. Ensure that entrepreneurs have access to “smart” capital: For a long time, Indian entrepreneurs have had little access to capital. It is true that in the last few years, several Venture Funds have entered the Indian market. And, while the sector is still in its infancy in India, VCs are providing capital as well as critical knowledge and access to potential partners, suppliers, and clients across the globe. However India has only a few angel investors who support an idea in the early stages before VCs become involved.
4. Enable networking and exchange: Entrepreneurs learn from experience - theirs and that of others. Much of the success of Indians in Silicon Valley is attributed to the experience, sharing and support TIE members have extended to young entrepreneurs.