Computational Biophysics blog

FRET Strangeness

2008-02-07T15:41:00.000-05:00

It strikes me how strange FRET is. A fluorophore (donor) is happily absorbing photons and sprays most back in all directions. If another fluorophore (acceptor -- whose absortion spectrum significantly overlaps the donor's emission spectrum) is brought nearby, this energy can be directed at the acceptor. Somehow the donor "knows" where the acceptor is and is communicating directly with it. There is certainly percedence for this in EM theory, but it still suprises me.

Letter to NYTimes

2008-01-21T23:36:00.001-05:00

I sent my first "letter to the editor" today in reference to an article on the recent Vytorin efficacy trials. See for yourself.

----

"What That Cholesterol Trial Didn't Show" complains about panic amongst patients after news spread of the trial results. But the article failed to point out that the Times itself carries some blame. The article published last week titled "Drug Has No Benefit in Trial,
Makers Say" states,

"This trial was designed to show that Zetia could reduce the growth of those plaques. Instead, the plaques actually grew almost twice as fast in patients taking Zetia along with Zocor than in those taking Zocor alone."

It should be clear why readers are alarmed. Whether the claim is true or not, the Times should discuss its involvement in the panic. I don't believe doing so will reflect poorly on the Times, quite the opposite.

Thanks for your time and great articles,

Daniel Terry
Student, Computational Biology & Medicine
Cornell University, Weill Medical College

Ubuntu 7.10

2007-11-03T23:52:00.000-04:00

I just upgraded Ubuntu 7.10 (Gutsy) a few days ago and found it awful. I've loved every version of Ubuntu I've used so far. First, Firefox was so slow as to be almost unusable. Scrolling or changing window size took several seconds to catch up. Adding the good old export MOZ_DISABLE_PANGO=1 to the top of /usr/bin/firefox did the trick as always. (Notably, this also made my fonts look much better!) I believe I've had this problem times before and had to resort to the same trick. Will Firefox always be so broken on install?? What's wrong with Pango anyway to make it so slow?? The CPU is constantly at 10% CPU (because of this bug).

Having fixed the former with a hack (as mentioned) and hoping for a kernel patch soon for the latter, I'm happy enough. But before I figured these out, I was planning to reinstall Fiesty. I can safely say this version is staying far away from my work machine.

Citation Management and Interfacing with RefWorks

2007-10-17T14:50:00.000-04:00

Background: Dr. Tung-Tien Sun of NYU gave a fantastic presentation last month teaching us poor souls how to do (experimental) science. I'd like to thank Dr. Sun for allowing grad students from surrounding universities to attend his talk, as otherwise I would've been left out. I recommend Dr. Sun's advice to every new scientist.

One of Dr. Sun's suggestions for writing papers is to keep a three-ring binder divided by subject area, with a page for each topic you work on. Within each section is a description of what's known, the central claims, central questions, etc with references to the relevant papers. There's also a protocols section so you can easily find citations for the protocols you use and the advantages of each -- even if you don't currently use them. Within this context, you also jot down you ideas, experiments to try, or the hidden information you see.

I see this as genius advice. I've never been able to keep straight the details of the papers I've read. Even putting down notes within my citation manager wasn't enough. So I've taken up this method, but in wiki form.

Problem: I want to implement the above three-ring binder as a wiki (as DokuWiki) with links to RefWorks so I can get at the paper/abstract/etc easily and also copy these into a paper for automated citation management. Each major field (smFRET techniques, photophysics, the ribosome, protocols) is a separate page, with subheadings for each topic.

Solution: The central problem is making it easy to cite papers quickly and be able to access them as a link. I use RefWorks as my citation manager (because it works with Linux), so I needed to interface DokuWiki with RefWorks.

Accessing RefWorks: DokuWiki needs to link directly into RefWorks. I accomplished this by sharing my account via RefShare and using the provided link. The format is:
http://www.refworks.com/refshare/?site=####/RWWS#####/#####&rn=4
The ###s in the middle are share-specific and the last number is the RefID number (4 here).

I created a DokuWiki Syntax model that replaces {{ID Desc.}} with a link to a my RefShare and that particular article. I can even replace a list as: {{ID Desc.; ID2 desc; ID3;}}. The best part is that I can copy these references directly into the paper as I'm writing and then use RefWorks to build the bibliography automatically -- it recognizes the format. The descriptions can be anything. I usually put the PI's name and year (Green&co 99) so I know who's lab it's from. Otherwise, I put the authors I know. This helps me remember the paper better.

The plugin (rename to .php) should be installed under dokuwiki/lib/plugins/refworks. You will have to change the RefShare address to your account in the code. In principle, I should set it up to use the DokuWiki configuration, but this works for now.

Firefox title attribute of anchor tag broken

2007-10-17T14:45:00.000-04:00

PROBLEM 1: Firefox 2.0 not displaying tooltips for anchor tags with the title attribute (when mouse hovers over link).

SOLUTION: The browser.chrome.toolbar_tips setting (in about:config) not only suppresses tooltips for the toolbar buttons in Firefox, but also title attribute tooltips as well (maybe disables all tooltips??). Reset this setting to its default (True) and it works.

The role of scientific blogging

2007-10-14T15:48:00.001-04:00

I'm wondering what the role of this blog should be in my conduct of research. I would love to write my thoughts on papers, give ideas on experiments to try, and even give some discussion on my current research.

Comments on Papers: I believe that this sort of discussion is great for science. The author can reply with further explanations or ideas, others can also post their thoughts, and the whole conversation is saved for all interested to read. Ideally, this should be attached to the paper itself on the publisher's website. Caveat: I will need to be careful that everything I say has a strong basis behind it. The last thing I want to do is insult a colleague because I misunderstood their work. I also worry I will give away my ideas for followup experiments I intend to do (see next point).

Ideas & Proposed Experiments: The obvious worry here is that someone else will run with your ideas and get there before you do. You are also unlikely to be cited. Should you not pursue the ideas, however, it may give those who come across it a window to a great discovery, which would otherwise be wasted. I suppose it would be wise of the author to comment on the post saying they are pursuing the work so that others don't produce redundant work and to encourage collaborations with those interested.

Discussing current research: At first it seems like an bad idea to release preliminary data, ideas, and current progress for the world to see, but isn't this what we already do when giving presentations? The only difference is the venue and the scope. A blog allows the same kind of discussion as a talk. Having a wider audience for your work doesn't seem to be a bad thing. I do wonder, however, if a publisher may refuse a paper if you've already "published" the data and some of the ideas online...

"Journal of Negative Results"

2007-10-13T20:25:00.000-04:00

Surprisingly, the fairytale journal we all joke about actually exists:

The Journal of Negative Results in Biomedical Science

I've always thought such a journal would be great to have. Evidently, Cornell has a subscription.

Politics in science

2007-10-07T21:08:00.000-04:00

I'm irritated with the way science works. I'm irritated with politics -- not the Washington, DC kind, but the kind that goes on between labs. A search for "politics and science", only tells me about stem sells, climate change, and such. I'd like to know more, but until I do, here are my thoughts.

I am a naive scientist. If someone asked me what all my reagents are, my protocols, raw data, and even my source code, I would be excited to give it to them. Why? Because it means it will be useful; it will produce scientific discovery! That's the point right? But such is career suicide. These things are highly-guarded secrets. Most labs won't let you touch any of the above unless you're a collaborator, and sometimes even then only if they are an author on all papers on the subject. It's madness.

The reason is simple. If your competitors got that information, they'll "beat" you. That is, they'll get the grants, they'll get the high profile papers (first), and ultimately they'll get the fame. Not one of those has anything to do with the core of scientific discovery: the furtherance of human knowledge. And yet they are not trivial concerns, at least not for students and new faculty.

Now competition is a good thing. It gives people the drive to produce quickly and innovatively. But this is madness. It's a problem when CryoEM people guard their maps as children, even if years have passed since they published a paper on it (publicly funded no less). All you get are their pretty (and tiny) pictures in the paper. If there's more to be learned there, you're on your own: make it yourself. Again. What an absurd and incredible waist of the time of brilliant scientists and precious little science funding.

Also a problem are the review papers whose purpose is only purpose is to flaunt the author's own research and show how "behind" everyone else in the field is. Or worse, purposefully refusing to cite their competitors. Papers become political swords instead of vehicles on insight.

I'm taking a course on research ethics now and it addresses some of these things. Our last speaker was David Skorton, the president of Cornell, who addressed who should be authors of a paper. We also covered conflict of interest, which can be a problem when your collaborator reviews your paper, steals the ideas therein, and rejects the paper (sadly this happens). But we aren't covering many of the relevant problems in the field. Is primary data-hiding responsible? Is it ok to "forget" to cite competitors work? Is it ok to demand authorship for the use of reagents? I don't think any of these practices are healthy for science. I think this atmosphere of closed and antagonistic research breads inefficiency and waste.

Final note: I'm not writing this because I was slighted or scooped, but because I am so dumbfounded at the politics in science. I'm deeply saddened at the prospect that I must adopt some of these practices to survive.

You don't know what you don't know

2007-09-30T11:52:00.000-04:00

In my experiments I often run across something that doesn't make any sense. Most likely it was a code bug, a misunderstanding about some aspect of the system, transposing numbers, or what have you. But it is very important to rely on your data because:

You don't know what you don't know

...as my advisor always says. I have definitely taken that statement to heart. If you have an outlier data point that is reproducible, you may have discovered something that will earn you fame or a Nobel prize. You never know enough about your system to be sure what the results should be. If you did, there's no point in doing the experiments to begin with.

This also emphasizes the importance of keeping good notes. This will help in make your results reproducible and allow you to understand why you get the strange results you do. For example: I heard a story about a student getting a very difficult experiment to work because she used tap water instead of ddH2O. But she did not document this because she did not want her mistake discovered (and did not want to repeat the experiment). Eventually they discovered what she had done differently and discovered some rare mineral was having an effect. Had she taken down all the minor deviations of the experiment, it would have been easy to deduce what was different.

Other examples include accidentally running a reaction too long or with too much of some ingredient and finding a pattern of increased yield. But if you don't document these minor things (and instead put what you were supposed to do), you will never know what made these reactions different.

Work hours

2007-08-07T21:46:00.000-04:00

A professor once told me that in his experience one must work at least 60 hrs/week to be successful. I have watched many profs come in very early and leave late at night every day, probably reaching 80 hours. I also see many postdocs who work from sunrise to midnight 6 days a week. I wonder how their marriages survive. I also can't help but wonder why admissions committees put such weight on "well-balanced" applicants who have lots of hobbies, community service, and social interests, when all of these appear completely incompatible with a career in academia. Is this what it really takes to be successful?

I know there are people who work more reasonable hours and are very successful. Productivity also plays a large role, since what's really important is what you produce, not how long you work. Indeed many that spend such long hours in the lab are simply error prone, probably because they're in so late.

Granted, if you love your work you will work long hours, and I do. Especially when programming, as I am now, I can pull more than 10 hours per day, plus 5 on the weekend. But going beyond 60 hrs per week is a sacrifice I'm not willing to make: I have other pursuits I can't sacrifice.

1330 First Ave.

2007-08-05T14:32:00.000-04:00

I just learned today that the building under construction at 71st and 72nd on first is actually housing for NewYork-Presbyterian hospital, the place where I work. The building is for doctors/staff, so we certainly won't be living there, but it may relieve some of the pressure from the current housing crunch.

Programming

2007-08-03T12:40:00.000-04:00

I am now in the habit of keeping a notebook at my desk whenever I'm working. On it I write a running set of ideas for how to solve the problems I run into when programming. When I'm having trouble solidifying an idea, I also write it out so that it becomes more formal and easier to implement. Whenever I reach a nice break point in my work, I also record a line that says "next step" with what I plan to do next so that I don't forget my train of thought after my break.

This mode of operation is working very well for me and I am finding myself much more productive.

With my current programming task, which involves modifying and improving existing matlab code, I am also finally implementing the strategies I have learned over the years.

Before starting, produce several example output files from existing code for comparison
Change as little as possible, even if it messy or nearly wrong
Unless absolutely necessary, choose the simplest, easiest to understand additions. Compute power is cheap, man hours are expensive.
Document as you go, rearranging functions to a logical order
Make only incremental changes to isolated parts of the program, at least at first
Do constant regression checking, including comparisons with "correct" output
Make constant (daily?) backups of your code so you can go back to working versions
At least 50% of "programming" time should be planning. I use a combination of notepaper and Backpackit.

Getting "Scooped"

2007-07-30T22:23:00.000-04:00

I'm very in to the idea of publishing thoughts, ideas for experiments, progress in current work, even data on a blog. The problem, as noted to me by a colleague today, is getting scooped. NatureJobs had an article about this problem; discussing how picking exciting, high-impact projects can be overly risky as a grad student. They also recommend choosing an advisor that is highly connected, so you know who your competitors are. I recommend reading the article, it had many great thoughts on the topic.

Despite the risk, scientists blog away, even publishing preliminary data. Some people are calling this grand idea Open Notebook Science, of which UsefulChemistry is a prime example. One blogger even earned an authorship from thoughts he published on his blog.

I don't think I'm brave enough, especially at this stage in my career, to follow such a paradigm, but I am very supportive of such efforts. In the mean time, I plan to post ideas, hypotheses, scientific critique, and some science commentary.

Obesity and gut bacteria: cause and effect?

2006-12-21T11:00:00.000-05:00

Two articles recently published in Nature described some very interesting experiments in obesity.

In the first (1), a sample of obese people and lean controls were sampled for gut flora levels. Obese patients showed 90% Firmicutes and 3% Bacteroidetes, while lean patients showed ~72% Firmicutes and ~25% Bacteroidetes (exact numbers surprisingly not published). Over the course of a year, the obese patients were put on a diet, during which time their gut flora levels became more like the lean patients'. By the end, the patients showed 73% Firmicutes and 15% Bacteroidetes.

In the second (2), two groups of flora-free mice were infected with obese and lean mice flora, respectively. The 'obese' group showed much more weight gain than the 'lean' group. The authors concluded from this result that obesity must in part be caused by differences in gut flora. This is possible since the normal energy balance is very tightly controlled and any perturbation, such as increased energy from the same food intake, would cause weight gain.

The interesting result here is that relative flora levels tracked the weight of the individuals, not the level or composition of food. It seems that the body is somehow regulating the levels of these colonies based on body mass (level of adioposity). My reading of this is that obese individuals, just like leptin knockout mice, erroneously feel they're starving. As a result, they harvest all the energy they can get, which then turns into fat. The mechanism could indeed include regulating the gut flora profile.

The writers of (1), most of the media world, and even other journals have assumed that the link is causative in the other direction: gut bacteria cause obesity, or at least influence its onset. The interpretation sells papers, but is not clear from the results. The assumption rises mostly out of the result that infecting flora-free mice with obese mouse flora causes them to gain more weight than those given lean mouse flora.

That the 'obese' mice gained more weight is not surprising, because Firmicutes (which the obese mice had more of) is more efficient at extracting energy. The key, however, is that this was recorded at only 2 weeks after the experiment. It is quite possible that bacterial levels would eventually go back to normal and the two groups would become identical. In fact if this was not the case, the changing gut flora profiles of the human subjects is very confusing.

The causal interpretation may well be correct, but I do not see enough evidence at this point to believe so. It is way too early to be championing this as a possible treatment for obesity.

[A few newspapers did report the findings correctly, though they were hard to find.]

----

1. Ley, R. E., Turnbaugh, P. J., Klein, S. & Gordon, J. I. Nature 444, 1022–1023 (2006).
2. Turnbaugh, P. J. et al. Nature 444, 1027–1031 (2006).

Dandelions

2006-12-18T15:16:00.000-05:00

To my surprise, dandelions are used as diuretics and were once used as a popular herbal remedy for many things. That's one reason they are so common: they were being harvested. Wikipedia has a list of uses for the weed.

One of the interesting properties of herbal remedies is that they contain a large cocktail of effectors and random molecules that medicine typically ignores as extraneous material. This molecule is purified and used as a drug. There are some instances I'm noticing where the herbal compound has components that offset or enhance the action of the primary drug. Diuretics cause loss of salts, particularly potassium, causing some side effects. Dandelion, also a diuretic, is very high in potassium, and so complements the active ingredient to give fewer side effects.

I wonder how much we are missing out on by focusing so much on single effector molecules when cocktails with simpler things may be more effective. Most drugs are single molecules and the rigors of clinical trials I assume inhibit making complex mixes of materials because they cannot be tuned if there is a problem. I do see mixes of already approved drugs, mostly in treating AIDS and cancer, but not overall. It also makes me wonder if we should be paying closer attention in the medical community to herbal treatments of simple conditions.

2006-10-25T12:22:00.000-04:00

Python in Biophysics:

...
Biskit: instructions for installing in Ubuntu
Change IFLAGS = -I$(PYTHON)/local/include/python$(VERSION)
to IFLAGS = -I$(PYTHON)/include/python$(VERSION)

Comment out CFLAGS = -Wall -w1 -O3 -tpp7 -vec_report0
Uncomment CFLAGS = -Wall -O3

python-biggles in Edgy-eft (maybe others) is broken. Download and compile it. Make sure to install gnuplot lib and libplot-dev. have to change makefile to refer to /usr/include instead of local.

INSTALLING surface racer:
download binary: http://monte.biochem.wisc.edu/~tsodikov/surface.html
copy files to ~/biskit/external/surface_racer_3/
change ~/biskit/external/defaults/surface_racer.dat
- change exe line to absolute path

First post.

2006-10-25T11:25:00.000-04:00

I'm going to hijack this blog for something different. Since most of my life right now is about computational biology and biophysics, I thought I'd write my thoughts, problems I've come across, and solutions I've designed so that others can get some use out of the work I've done. Everyone uses papers heavily enough for information, techniques, etc, but there is so much more that is just snippets of solutions and ideas that aren't being noticed by most researches because they are only available in private discussions.

TV Listings in Google Personalized Home & Calendar

2006-06-22T01:13:00.000-04:00

EvokeTV is a nifty TV listing manager that lets you select certain shows you want to see in the future and tells you when they pop up. It's in starting stages now but promises to bring quite a selection of features to managing your TV viewing habbits. Its purpose is aparently building communities around TV viewing habbits. Its also free and ad-free. But there's more.

To do either of the two tasks below, follow these steps first. Create an account at EvokeTV.com and login. You will need to find the shows you want to show up in your listings and give them a rating. You can set thresholds for how highly-rated a show must be to show up in your listings.

Adding custom TV listings on your Google Personal Homepage
In EvokeTV, click on "My Schedule", select a rating threshold, and copy the address for the RSS feed. Now open up your Google Personalized Home and click "Add Content" at the top-left of the page. Click on "Add by URL" (to the right of the search bar button). And there it is.

Adding custom TV listings to Google Calendar
In EvokeTV, click on "My Schedule", select a rating threshold, and copy the address of the "Subscribe to iCal Feed" link. Now open up Google Calendar and click on the + next to "Other Calendars". Select "Public Calendar Address" and enter the iCal feed address you copied. The address will start with webcal:// which must be changed to http:// for it to work. Hit "add" and you should now have your TV listings in your calendar!

[Ed: since this time, EvokeTV has gone dark]

GMail

2006-06-19T02:30:00.000-04:00

I'm moving all my email to my GMail account. I finally decided to consolidate everything to gmail. I was sick of managing all the emails from all 6 of my email accounts, sick of wierd warnings about bad Certificates from one host, and sick of waiting for Thunderbird to load, and its various bugs at the moment (constantly consuming 20% CPU). What did it was my finding the ability to send email from any of my addresses and, mostly importantly, have replies come from the address the person at the other end sent it to. That way I can keep professional corrospondance on my IUPUI account seperate and not have to tell people about my gmail address. All without ever using all those accounts directly again. Ohsonice.

I also discovered Google Analytics and have signed up for an account to replace my truely ancient eXTReMe trackers. Funny how their website has changed in like a decade. Amazingly its free and super high quality...hope I won't have to wait too long to get an account.

Google really is taking over the world. Gmail is my only mail interface, Google Home is my homepage, its the only search engine I use, I use google maps/Earth constantly, and have now started using the caldenar as well. I know their setup isn't suitable to everyone, but its perfect for me. It has most everything I need all in one place. So even if their service isn't /quite/ as good, I'l probably still use it anyway, because of the shear convenience.

I have only one problem with the whole mess. Google is storing a huge amount of my personal information: all my emails, my chats, even my search history. Stored on my desktop, its not hard to encrypt my data soas to be impossible for the gov or anyone else to get their hands on. But with Google a .gov supeana is all it takes to get them all. I know Google has fought for user privacy in the past, but I'm not sure that will always be possible. I also worry about hackers. There's too much trust in this structure for my liking. Yet I still enable all these features and use them.

Wish list manager

2006-06-16T21:08:00.000-04:00

I need the following:

A wish list manager that will keep a database of all songs you've marked as wanting to buy, along with a rating for how good it is (or how much you want to buy it). You will then be able to see which albums to buy next that have the most highly rated songs. You'l have an idea of which songs to buy next out of the enormous list. Perhaps the program could even suggest other artists not only based on what you have purchased (and have on your HD), but also based on what you want.

In addition, you can select a download service (EMusic, iTMS, Napster) and it will show you which songs are available through that service. You could also get warnings when any of your songs becomes available on a particular service, or any at all. For example, I want whenever possible to buy my music from no-DRM stores, or those with higher bitrates instead of iTMS. But it takes major time to search for each one from all the different services, and it could be added the next day, so you have to search over and over. This is impossible for a person.

The app would be internet based, with a central server. That way searching for songs through all the record databases only need be done once if it is found. If it were a desktop (decentralized) app, one song may be done maaaany times, flooding the servers.

I'm not sure how easy it is to search the catalogues of all the different sites, although MP3.com seems to do it, but for only a few services and not their entire catalogue (aparently). I think this would be the most difficult part. Its possible to just hack the backend for the client, or even highjack a client and run searches through that, but it seems to be a pretty inefficient way that could get your account locked because of the activity levels...

The idea should be profitable through referals when people buy songs through the website/software. I'd think about doing it except there's no way I have time.

Product Activation

2006-06-09T14:47:00.000-04:00

If I invite some friends over to watch House MD, my TV doesn't complain that it's only licensed for 2 simulateous users. It doesn't lock down and request I buy a site license if I allow someone other than myself to use the remote. My pictures don't restrict me from printing them only on "approved" printers. My toaster does require reactivation if I move it to a different room. I don't need to call Chevrolet to authorize the New Vehical Configuration if I change my tire. Why is it that we accept such restrictive, arcane, and downright anti-consumer schemes in software?

I switched away from Windows XP partly because of its new product activation scheme. My legit copy of XP Pro eventually decided I shouldn't have access to it. Why? Because I upgraded my RAM and changed my video card. For that heinous mistake, I had to spend over half an hour on the phone to India unlocking software I purchased! It's hard for me to see why I as a customer should be locked out of a product I legally own. Because I am a computer enthusiast, trying new hardware all the time, I'm constantly locked out. After the third time of having to call Microsoft to beg for permission to use my own property, I threw the disk in a corner and converted completely to Linux on all my machines.

Yesterday I had a similar experience with Photoshop. I purchased Creative Suite 2, mostly for Photoshop and Illustrator. I quickly learned that it too required activation, and could only be installed on 2 computers. This meant I was stuck just putting it on my desktop (through QEmu and the old XP install), and couldn't use it on my laptop, work machine, or spare. And should I make the mistake of having my hard drive crash or be stolen, I was out of luck for the license. You can transfer authorization between computers, but only if you have access to them. It doesn't matter that you still have the actual software (CDs and license) in hand. As it turns out, you can even lose activation through hardware changes or writes to certain parts of the HD. So while out on the road you could randomly be locked out of your own software and have to call and beg Adobe for permission to use software you just paid hundreds of dollars for.

And finally, I purchased this book, which came with Wavefunction's Spartan chemical modeling software needed for class. Turns out, you can only install this software once. That's it. Want to transfer it to another machine? Nope. Hard drive crash? Sorry. And that's what happened. Not long after activating it, my laptop's install was toasted and I had to reformat. I contacted Wavefunction, and they released the key so I could activate again, but any more and I've have to pay for a new license.

This last case illustrates one of the major problems with activation schemes. I'm no longer assured that I will be able to use software I purchase. It may well decide someday to lock me out, maybe due to a software error, maybe due to overzealous locking, or maybe because the company goes out of business, or stops supporting the software. Then the software I paid for is worthless. This is simply unacceptable as a consumer. If I buy a product, I expect it to be my own, and not under the control of fickle software corporations.

Compression

2006-06-08T23:44:00.000-04:00

For those not away, the people behind StuffIt have integrated a special algorithm for compressing JPEG images that gives 20-30% compression on average to files that by all other algorithms had 1% or less. When I first heard about (many monthes ago), I thought the reviewer had made a mistake, or that it had been manufactured by Alladin for hype. But not much later, I found it was true.

Their technique turns out to be quite innovative and incredibly simple. JPEG compression works by degrading an image and then running a DCT transform. This only takes care of local areas of similar color, but not overall compression. The end data still has a fair bit of entropy, so it can be compressed much further. JPEG standard uses a simple Huffman coding, which is good for compressing data that only uses a subset of its character space, but terrible for finding patterns or repeats.

The use of this algorithm itself is the problem. All but the most sophisticated compressors (PPMs, etc) are stream-based algorithms that essentially keep a dictionary of repeated charactes (LZW) or blocks (BTW). Rare items have large encodings (which is ok since they appear infrequently) while common items have very short encodings. An algorithm like Huffman coding essentially scrambles the data in such a way to make it impossible for a stream compressor to find patterns.

StuffIt takes the very simple step of removing this obstacle by recognizing that a file is a JPEG, reading its headers, and decoding it; giving the raw DCT output. It then uses more modern compression techniques, producing a new file. The actual algorithm used hasn't been released (to my knowledge), but it could well be something as simple as BTW or as complicated as PPM. When a file needs to be decompressed, it is then uncompressed to the raw DCT data, then re-encoded using JPEG-standard Huffman coding, producing a bit-for-bit identical copy.

I have heard that StuffIt uses similar techniques on other file types, but I dont know the details.

I would be interested to decode a JPEG (to its DCT data) and then try the various compression algorithms to see which StuffIt is using, and perhaps find out what the true limits of JPEG lossless compression are.

---

The simple innovation here is the idea of not simply producing an encoding for a bitstream, but running intelligent, complex transforms on it to make it more compressible to standard algorithms. The same could no doubt be done for many other formats.

It seems, then, that there is a great market for compression programs that can detect the data type, tag it, and use different algorithms. Few programs I know of do this. It'd be especially nice to have a Linux command line tool that does just that.

One example I thought of last night was to make an ISO image compressor, which would mount the image and save the filesystem. It would then reorder the image, putting data of similar types together. For example, it would group executable binaries (.dll, .exe) in one area, and .jpeg images in another. You could then use specialized algorithms that work best for each dataset. Grouping them as such will also make more efficient use of dictionary-based compressors. Perhaps it could even look for LZW/DWT compressed files, inflate, and then recompress them with something more suitable.

Perhaps it could even be aware of common features of ISO images, and only store differences from this basic template in the stored file. Such an idea could be used in may scenarios. If a certain header is common in many DLL files, why not store it with the compressor and just keep the differences in the compressed file?

It makes you wonder if they will find a similar technique to the JPEG recomressor I first mentioned for other files types, maybe MP3s, MPEG videos, or executables. Its all about making the data more apetizing to the algorithms we already have.

Problems with Emryonic Genes

2005-10-13T11:50:00.000-05:00

Too much information can be a bad thing, especially if you have cancer.

The germ line DNA human genome is kept intact in all without any modifications from birth (with one exception: B & T cells). Tags and markers that up/down regulate the expression of genes does indeed change, but the actual data is left intact. This leads to an interesting situation. Skin cells know how to make a brain. Heart cells know how to make a spleen. In fact, they all know how to make another human being completely.

This fact is interesting because one would assume that as an animal evolves, it should at some point do some house cleaning on that huge genome. But yet they don't. It is far more common to see an animal with a set of genes duplicated in several areas (by earlier mutations). This is how the TCR was born from the BCR genes (though I'm not positive it went in that order). Some fungi even have mutations that cause such a large repeated segment to occur that they can't survive. Yet animals with random *losses* of genes are rare.

But all that information can be deadly. My best example is that of human melanomas, which metastasize and spread faster than any other cancer. Researchers have found that this is a result of these cells activating a gene that is used by the developing embryo to shoot skin cells to the outer surface where it develops into the skin. Melanomas reactivate this gene and use its instructions to spread. Other cancers which don't have this advantage can take many years (if ever) to metastasize.

Had these cells deleted this unneeded information at birth, the cancers would not have metastasized so easily. In fact, this goes to the heart of information security: instructions should be on a need to know basis only, and those things no longer needed should be revoked ASAP. But the human body undergoes no such genetic changes, though it easily could.

All that extra information also has another effect. Say 50% of the DNA in a particular cell is not needed because it is never activated. If that 50% is removed, it would also decrease the surface area of the nucleus, which means that radiation is slightly less likely to hit the nucleus of any cell, especially of lower wavelength waves. Much more importantly, the decrease in the number of genes gives less chance for a mutation during cell division.

That being the case, it is hard to see why nature has not found this solution and tried to reduce the amount of DNA floating around in the somatic line. If it isn't needed why keep it? If the immune system could evolve the ability to make these deletions, why no the rest?? This behavior was selected because with allowing accidental duplications of genes, the instructions for building one mechanism could be transported as the basis of a new mechanism without the need to evolve one from scratch. Yet the risks associated with too much data are usually not a problem. Organisms have gotten very good of keeping track and protecting all that data with little issue. Especially since most organisms don't live very long, cancers and mutations are not such a big deal.

But to us, where every life is very important, I suppose we would want to stop that trend, maybe even reverse it, for the sake of individual life.

Scratches

2005-10-09T00:47:00.000-05:00

Reading through a friend's profile, I noticed a little pink bar randomly in a bunch of blank space. What was that? Probably a remnant from a long gone message scribbled in its pages forever lost. But there was a piece left. One can sit and wonder about what was once there. But really, that's one of the sad things about our digital world. Usually, nothing remains when something is removed. People don't leave marks as they traverse its hallways.

Think about it for a moment. When something is erased from this world, it always leaves traces. Endentations in notebooks, chalk smears, marker smears, candy wrappers, and empty water bottles. There's always going to be something left behind to show it was ever there. Every change leaves its mark, even if only in the form of a small scratch on the floor you made as you left.

A Need for Isolated User Accounts

2005-10-06T13:01:00.000-05:00

Most average computer users, especially those less technically-inclined, have loads of spyware slowing down their machine. It gets there both from users mistakenly installing malicious software they think they have to install, or as a bundled package with other software. In the second, the software is usually borderline illegal, such as Peer-to-Peer network software (KaZaa, Morpheus, etc).

The problem I am starting to see is that in households with 1 computer and many users, the kids end up installing this software, and everyone else has to deal with it. The parents can be told "don't download X software" 'til the end of time, but it does not solve the problem.

Most would suggest a locked down account for the problem users. Such accounts are avoided because it is difficult to make limited accounts for non-Pro users. In addition, these users cannot install their own software, which could be a problem. What if little Johnny needs to install a program for school? Parents would generally just let them use their unrestricted account or take away the restrictions, nullifying the security.

In such a situation, the idea is to have programs install only for the current user, and no one else. The base system can still be accessible to all users, including a variety of commonly-used software (office apps, browser, etc). Quite simply, if a user installs a program, it cannot be run on the other accounts. This way if one users manages to totally mess up their account, other users are left unaffected. Of course there is still need for an administrative account to install new base software and do updates. Forcing users to only use this when they have to is important.

But this type of setup isn't possible with most operating systems today. On Windows, limited accounts break many programs to begin with, and there is no way to limit a program to one account. On Linux, the same problem. Users cannot really install software for only their account.

This needs to change. I'm sure users would far rather just nuke Johnny's account than the entire machine, or buy a new one.

Hybridized Immunity

2005-09-02T23:28:00.000-05:00

I think I should made a section on my website for research into more bizarre concepts. They may not be possible today because of technical restrictions but may be possible in the (perhaps distant) future. Or maybe they'll never be implemented for ethical or other reasons. My first example: improving the immune system.

The human immune system is amazingly beautiful and complex, accomplishing its goals remarkably well. But it isn't perfect. As it is, specific immunity is mediated by randomly generating antibodies and TCRs, then magnifying those that are most reactive to pathogen when exposed. During maturation, over 98% of the generated cells apoptos usually for being self-reactive, a very wasteful process indeed. Even worse, this process must be redone for every person on earth.

I propose a hybridized system using computers, which are more optimal for this situation. When a new pathogen is discovered, antibody and/or TCR can be calculated that are minimally self-reactive, maximally target antigen-reactive, and highly specific. Once generated, the information can be disseminated over the internet to all people. A machine can then use this information to create individually genetically modified mature B and T cells with the appropriate MHC for immunization against the target pathogen. To be effective, the B cells would have to be made to differentiate into a large number of the long lived memory cells for life long immunity. Exposure to faux-pathogen in vitro would do the trick.

Once injected, the patient has perfect immunity to a pathogen he/she has never encountered. There is no risk of autoimmune problems as there is with vaccine since self-reactivity is highly scrutinized during generation. In addition, there is no chance the host could become infected with the pathogen (as with live vaccine), and no chance of an alergic reaction (these are just modified host cells).

There is one major caveat to this system: different people have different self antigens. This problem causes transplant tissue rejection as the immune system recognizes the tissue as non-self and destroys it. In the hybrid immune system, care must be taken to catalogue all known human antigens for testing. This may be a problem if some of these self-antigens are necessary for identification of some pathogen (these rare individuals, if they exist, would simply not be immunized against those specific pathogens). A nice side effect of this: anyone using this system with their real acquired immune system disabled will be a universal tissue reciever, baring graft-versus-host disease (bone marrow).

One other note: persons using this as treatment for autoimmunity would need to have their real adaptive immune system disabled to prevent the self-reactive antibody/TCR from being created. They may also need a plasma exchange to remove the old antibodies. Drugs that would be needed for suppressing B and T cell creation and maturation should be easy to make.

The system described sounds great, but isn't possible today. Such a system requires machines that can modify a patient's cell's genetic information completely accurately 100% of the time with *low cost*. This simply isn't possibly today. In fact, it may be many decades before this is feasible.

[For those that apparently didn't get it, this post was mostly a joke.]

The Future of Pharmacology

2005-08-24T00:33:00.000-05:00

In the last month or so, I've become interested in a new field called Computational Biology. Its a large, interdisciplinary field for both biology and computer science. Basically, it is the study of using computers to study (or predict) biological processes).

This is a significant field with some fantastic possibilities. For example, the pharmaceutical industry could get a huge boost here (and in fact already are using it a bit). Computers could be used to simulate the pharmakinetics of a drug when administered to a living patient. The interactions could be simulated over a wide range of possibilities, various ages, sexes, genetic disorders, diseases, allergies, etc. Realistically, the human body is just a big machine, so it can be simulated quite accurately by knowing the mathematics behind the chemistry involved.

If such a simulated clinical trial could be completed, it would allow drugs to come to market far faster, assuming enough computer power. Potentially also, far far cheaper, again depending on computer power. Not doing them on human patients saves some possible nasty side effects rare patients might have. Done correctly, it would also catch almost all rare and very long term side effects that real clinical trials might miss.

Of course that all assumes the simulation is perfect, which with current algorithms, it most certainly isn't. It also assumes that there is enough computational power to make such enormous computations. But this isn't available either. But unlike human scientists and living systems, computers get significantly faster every year. We may be talking about computers in 50 or 100 years, but it's still in our future.

Perhaps more immediately, biochemical computer simulations could allow us to find a wide array of drug candidates. Most today are found by exhaustive search (which can take forever to test only a few for a few, and the potential may be missed), or by random chance. Even with todays computing power on large distributed grids, drug candidates can be tested for low activating energy "hits" which could be good drug candidates for a particular condition. This too requires a great deal of computer power, but it is within the next few years to be very useful, unlike simulated clinical trials.

Another area of study that is actively unfolding is understanding various diseases and problems through mapping the human genom and understanding the purpose of each gene inside. Mapping it completely will allow for easier diagnosis of a wide range of genetic problems. It can also help us understand pathogens like viruses and bacteria by unraveling their RNA/DNA.

Looking at those problems, it makes me laugh when I hear people saying "nobody could possibly need more CPU power than we have today". People ignore problems solvable with huge computational power because such power isn't available, and then assume more power has no use. Fortunately, there is still a lot of demand for more power in other fields that will continue to advance available power. I'l keep my eye on the inovations upcoming in generations of computer power to come.

Manifests for People

2005-08-10T21:03:00.000-05:00

On today's platforms, running programs requires a lot of trust on the part of the user. Once run, programs have complete control over all resources that user has access to. This bad policy is reminiscent of the early mainframe era where everyone had access to each other's user accounts, relying on trust to keep the system secure. But trust is an even worse proposition when dealing with unknown programs floating around the web.

But why must it be that way? We've had file permissions for decades now, why can't the same thing be implemented for programs?

One way to mitigate this problem is through an old technique: manifests. Traditional manifests are only a small list of files a program is allowed to install. A more useful manifest would be one that details each and every action a program may take. What files can it access? Can it overwrite system files? Can it connect to the internet?

Such a system could be implemented similarly to the old file permissions, either by bit fields in the filesystem, or some sort of access control list. The kernel could then check these permissions before granting access to certain actions.

In a world where this type of system is implemented, programs would come with a manifest listing the actions it wishes to have permission to perform. Users would be presented with these abilities in a list before the program is allowed to first run. They cancel out if the requested abilities seemed excessive. Users would also be allowed to uncheck certain abilities. The program might crash, but at least no harm could be done.

By far, however, the most difficult part in implementing this system would be making sure it would be used by users. Abilities would have to be chosen very carefully so that they clearly define a barrier of activity that is or is not allowed. There should be no way to get around the spirit of what the user wishes. In addition, the abilities need to be described in a way the user can clearly understand. For instance, if the user is presented with the ability "Trace OS key events", the user may simply allow it, not knowing he or she is installing a key logger.

Making these abilities as descriptive as possible would not be easy because the concepts are so abstracted from the OS implementation. Some things like "Access the internet" are easy. Others, like "Display ads", are not. Implementing these might take some effort. The ability "Send email", for example, could be tracked by the user of port 25 (SMTP) or traffic sniffers watching for SMTP headers. Logging keystrokes or monitoring other programs could be prevented by restricting access to certain system calls and interfaces.

The actual number of abilities/permissions needs to be small. A situation where the user is confronted with a long list of needed permissions would cause users to be "OK"-click-happy. Writers of dubious software are well aware of this problem. They write immense, overly-technical EULAs for this reason; nobody will read them.

Lastly, choosing the right abilities for todays security scenario is important for them to work properly. But this changes over the years. Whatever permissions system is setup should somehow be adaptable and updatable over time to counter new threats and technologies. This should be done while also remembering to always keep the number of permissions minimal to prevent the above problem.

Grad School - The State of Computer Security

2005-08-06T12:13:00.000-05:00

I will be applying for graduate school soon, with the hopes of getting a Ph.D in something related to computer security, probably with some focuses on usability.

Basically, I want to solve outstanding problems in our computer society causing the general public some trauma: viruses, trojans, spyware, malicious adware, phishing, identity theft, information theft, and other security exploits. Regular people are becoming increasingly frustrated with the problems these have caused. Productivity, time, and money are being wasted. My goal is not to make anti-virus software, it is to fundamentally change or solve these problems by approaching them in completely different directions.

Right now I only have some minor ideas for solving these, but that's what grad school is for.

Previously, the solution to most of these problems was a case-by-case approach in which a company evaluated new threats each day, releasing some sort of patch to be distributed amongst clients. Clearly this is a kludge approach, there should be other solutions that get to the root of the problem. The goal is to create a program that protects against most attacks, previous and future, with no updates. This certainly isn't easy.

I intend to devote much more attention here to these problems, including potential solutions. I will also keep you posted on how my applications are going.

Federal Real-ID Bill

2005-05-12T11:08:00.000-05:00

Today congress finally passed the Real-ID Act in the form of a rider on an emergency defense bill no sane rep would vote against (100-0). There's been quite a bit of controversy over this particular bill. The goal is clearly to standardize all state driver's licenses with the hopes of one day having a true National ID Card. A fury is rising about this bill on the 'net, but I have heard little mention in the MSM. But Real-ID should be seen as less of a threat and more of a wake-up call.

Bloggers in the tech industry are going crazy over this. "Gold mine for identity thieves." To them I say, you're fooling yourselves if you think it will be any easier with a central government ID database than it is today. The government database will be heavily secured. Attacks go for the easiest target, and those are already out there: agencies like Lexis Nexis and ChoicePoint. Both of these companies have had massive breaches where thieves gained hundreds of thousands of records. And the reports keep rolling in. Your data isn't safe, but Real-ID has nothing to do with it.

Republicans are going nuts over the idea of a federal id card as well, for privacy and paranoia reasons. All of this is naive. We already have a federal ID card, its called the Driver's License. What few never had one still had to pay taxes. You're ID number is either your Social Security number of Tax ID. They still know who you are. Many republicans argue that allowing the feds to consolidate all this information is dangerous. But really, they can already purchase this information themselves from a data mining company like Lexis Nexis or Choice Point (among many others). The only thing that changes is the government no longer has to pay off an outside firm to do its spying. Your anonymity is gone, but Real-ID has nothing to do with it.

Anonymity before the government is a quaint thing, its been gone for many years. Real-ID should make us all stop and think about where the real threats are coming from. We as the tech industry shouldn't be fighting against Real-ID, but rather the underlying problems (data mining companies, lax security).

Advertising works

2005-04-27T03:39:00.000-05:00

It is not possible to compare and scrutinize every product we purchase. We can't weigh the values and history of every brand, the source of their labor, the source of their ingrediants, the politics of their CEOs, or even simply the quality of the product. Who has time to check reviews and back panels of every loaf of bread, mechanical pencil, and stack of paper? This is exactly why advertising works.

Usability Concerns in Modern Software

2005-03-09T22:08:00.000-05:00

Or maybe I should say "things that really irritate me" in software and web apps I use day-to-day.

Don't Get in the User's Way
I believe probably one of the most important rules in usability should be "don't get in the user's way". I've heard this repeated over and over, especially by Linux users. Many times I find myself fighting the OS to do the simplest tasks. Programs will make it a policy to interrupt me in the middle of typing with some urgent message, like "I'm done checking your mail." They pop up and get in your way at the most inopportune times. They just make your life difficult.

The first I've noticed is the Tip of the Day dialog that most Linux apps seem to have. While it might be a nice tool (does anybody use them? I never do), they block your way from using the app you just launched. You opened up an editor to edit something. If I want help, I'll ask for it, don't shove it down my throat. Thankfully they often have "never show this again" checkboxes.

Pointless Dialogs
Another is common in nearly all text apps is the "No results found" message box. The problem here is that if I'm looking for a list of works, I have to click "ok" every single time I want to do another search. The dialog gets in my way. It should just show a little bar and play a sound to say "no results". But, unfortunately, few do this. One of the few apps I've seen implement this properly is FireFox. The message box forces the user to move over to the mouse, slowing him/her down with every attempted search. This can get very frustrating.

Blocking the User
A related problem is the apparent obsession with modal progress dialogs. It makes no sense to block the user from using your app just because some background service (like categorizing mail, adding files, downloading) is running. The user should be free to use the rest of the app. Putting up a modal dialog only gets in the way of a user. The correct solution would be to put a small progress bar and message at the bottom of the screen.

I don't see the above too much, but what I do frequently see is a program with a model dialog (typically for CD burning, searching, or preferences) that blocks the parent window from being minimized or resized. This usually means I cant get at my desktop at all. And in some extreme cases with System model dialogs, I can't even get at other apps. In the case of CD burning, I can't simply stop the burn. And I shouldn't have to quit out of the modal dialog just to get that window out of the way. I know the window needs to be in front of its parents, but it doesn't need to take over my desktop. Again "stop getting in my way".

Performing Pointless Repetitive Actions
This is a big grip of mine that has gotten better in recent years. If an action can be automated it should be. There's no point in the user sitting at a computer hitting "Yes, Overwrite" or "No, don't save" thousands of times just to perform a batch action. The "yes/no to all" checkbox is a marvelous invention here, but sadly it's missing in important places.

Try opening a bunch of raw images in photoshop. All you wanted to do was view them. Upon exit, it wants to save all of them, even if you opened hundreds, since they were not in a "normal" format. So hundreds of "do you want to save" dialogs pop up, one after another, forcing me to click "no" for each and every one. Why is there no "Yes/No to All" button here? Can't the computer do all this clicking for me?

Assuming too much about your user (web)
I'm starting to get frustrated with webmasters who create sites that are hard boxed into 1600x1200 tables. Now...not all of us have such a high resolution, in fact few of us do. And those of us who do frankly don't want to read an entire book in one line. Why do this? It makes no sense at all, it only frustrates users. In most cases, tables should be variable size so the user can choose how big to make them. And in cases where squaring it is a must, choose smaller resolutions that you can assume your users will have, like 1027x768.

Consistency
Users are used to things being done a certain way. The OK and Cancel buttons are generally in the same place. The menus are at the top. New, Open and Save are under the File menu. Etc. But there are some rebel apps out there that strike out against the expected, much to the pain of their users. Take GIMP. I've yet to see a new user ever figure out how to save files on their own. "Wait, there's no Save under File...huuuh?" Why are menus done in such an unusual way in GIMP? Maybe it made sense to the creators, but it makes no sense to anyone who's been using computers for awhile.

Assuming "Programmer Knows Best"
Definitely going to pick on the GIMP people again here. Why does GIMP still, to this day, not have an MDI interface, even as an option? Nearly all other image programs have such an option (usually, its the default). Yet the GIMP team still refuses to accept their user's outcry and forces them to use the clunking method huge numbers of windows spread across your screen, cluttering your taskbar. I've heard many explanations for this, but none that make sense. Even if you, the developer, believe organization is "the window manager's job", just make it an option. Its not that hard. You may like it one way, but let the user's have their pick. Isn't that what Linux and Open Source is all about?

Easy CD Creator has a problem somewhat like this. When burning a CD, the user must know before the first burn exactly how many disks will be made of that particular session. Should you realize you want one more you're out of luck. As soon as the burn(s) finish, the project is closed (at least it asks you to save it..). What if I wanted to burn more copies? I know I "did wrong" by not telling you exactly what I wanted before hand. But is it so bad to change your mind? Programmers should design for easy of use, not for what "only makes sense".

I think its safe to sum this entire post up in just a few words: don't get in the way of your users, make work go as quickly as possible, make good interfaces, and give your user's choice. If we could get more apps written for this mantra, we'd save ourselves many headaches.

Usability Issues in Linux

2005-02-15T01:27:00.000-05:00

Or rather "Linux-based distros". So many distros, but they all fall short. They are all just so hard to use, so complex. They fail so miserably at some of the simplest tasks. Linux is close to being ready for the desktop, but it still has some issues.

The first problem for so long has been the packaging system. Quite frankly since the beginning these have been outright horrible. Just thinking of the dependency nightmares of the old up2date and Mandrake Update systems gives me nightmares. Things are getting better with Portage, Xandros Networks, and even Click-N-Run, but there's still serious issues.

Let me ask you a question. When you open up "Add Remove Programs" to uninstall something, how many icons do you see? Less than 20. When you try to uninstall something in most package system, how many? Thousands, it seems. Systems that categorize this help tremendously, but there's still so many of them. Major programs should be separated somehow from the huge number of drivers, OS utilities, and random things like fonts. Programs need to be easy to uninstall.

Let me ask you yet another question. When you install a Windows app, can you immediately run it? Or to be specific, can you find it? Of course! It's always in your start menu. On Linux, I never know. Right now I still can't figure out where pptp-gtk-config went after I rpm-ivh'd it. All I know is it's not in my path... Same goes with various major GUI programs like XMMS, which get tossed all over the HD with no clue how to find it. Programmers expect us to know to type "xmms" at the prompt, but that's cumbersome, and as with pptp-gtk-config, the name of the package is different than the name of the program. How to do you know what to type? It shouldn't be this hard.

The solution is simple. Make an area, like the start menu, to put links to installed programs in that is standardized across Linux. It doesn't have to be the start menu, it can just be an easily accessible folder. Just let me know where in the world they went so I can RUN them. It seems like something I shouldn't have to complain about.

Still more usability issues. In windows, how hard is it to get yourself on the company VPN? Easy. New Connection->VPN->server name & login->connect. What about Linux? Don't get me started about having to patch my kernel with MPPE, get pptp, patch it with MPPE, recompile and install my kernel, compile and install pptp, and then enough config'ing and route'ing to make me sick (and it still doesn't work). Again, why so hard? Such simple things should be build into a distro, just like windows. They're just too common. And, frankly, too easy to have done the right way to begin with.

A related problem is that I honestly have no idea as to the state of my internet connections when I'm on Linux. I have to ifconfig/iwconfig to find out. Constantly. Why is there no little blinking computer in my systray to tell me these things? Again, such a simple problem.

What bothers me is that instead of addressing these issues by making the little pieces to make a system "just work" and be easy to use, distro makers just pack more and more bloat into their systems in the form of a billion choices of every conceivable task. Honestly, I could care less that I can listen to a song in no less than 6 installed players, but rather that I can easily get online.

This is the point of a distro: to bring together all the drivers, kernel patches, basic system components, and helpers (like package managers), and system utilities (connection manager) to create a system that works out of the box. I know all the apps to make the perfect distro are out there. But it shouldn't be up to me to get them, that's what distros are for. More and more, they're losing this focus. Pouring effort into so many apps, but missing so many basic and necessary system utilities.

Distro makers have no focus on the real problem: making it all "just work". We still have a long way to go.

Why DRM Isn't So Bad (Part 2)

2005-01-29T03:42:00.000-05:00

So often today, all I hear about is how the virus that is DRM is spreading throughout the consumer media space. About how our rights are being taken away, and how evil DRM is. Why such a backlash? Isn't there anything good about the possibility of a DRM future?

DRM actually accomplishes a task of such importance that so many people fail to realize. And that is to make an intangible object of which "ownership" is impossible, into a tangible. Consumers want to way to say "yes, I bought this," its mine. They want a way to prove to themselves that what they just got was more like a CD or DVD. A tangible. DRM accomplishes this task by making a format that is specialized to the user. Only the owner can play the file. Without this feeling, customers would always feel better about buying the real media. This solves a serious problem.

DRM has a more important role though. Even though we'd like to go stay in the days of CDs and good old analog TV where there were no restrictions, media companies don't. The only way we're going to have a future of digital convergence is if the media companies can feel warm and fuzzy about the safety of the content the release. That means no copying and sharing. Once this is secured, media companies will finally be willing to invest their catalogues in online digital media.

One day we will see a time where anyone can sit at their TV, buy a few movies, rent a couple others, and watch them all without ever leaving their couch for a wasteful drive to BlockBuster. This is the future. But its only possible if content owners are in on it. And there's only one way they will be: if they have control. Like it or not, that means our digital future will be a DRM'd future.

Why DRM Is So Bad (Part 1)

2005-01-29T02:55:00.000-05:00

So often today, all I hear about is how the virus that is DRM is spreading throughout the consumer media space. About how our rights are being taken away, and how evil DRM is. Why are so many people so upset at DRM? It's not Digital Rights Management they fear, its Digital Restrictions Management. Unfortunately, at the same time media companies want to prevent rampant piracy of their works, they also want to stop many legal uses. Many worry that if DRM becomes mainstream, big media will lock them out of their fair use rights. Or worse, try to force them into bizarre "rental" models for all media.

Some in the software industry brush this off as irrational paranoia. "It will all work out fine, its just giving the artists choices." Maybe if media companies were all benevolent providers with only good intentions, but we all know that's not always the case. I give you the mobile phone giant Verizon. This company has mastered the technique so many in the tech industry fear: complete customer control through DRM. How do they do it?

Verizon makes its money off its network. When customers take hundreds of photos on their brand new Motorola phone, do you think Verizon is ok with them just plugging it into their PC to store them? Of course not! There's no money in that. Instead, Verizon has a deal with Motorola to cripple the phones they sell to block any way of transferring out media files (like pictures and movies) except over their network. And they'll be sure to charge you handily for every byte of it.

Take another example. Ever notice the companies advertising cell phone wallpapers and ringtones if you "just call this number.."? Verizon knows better than to allow their customers to directly upload wallpapers and ringtones to their phones from their PCs, or worse, make their own. Once again, there's no profit in that! Instead Verizon locks its consumers into only getting these features over their network. And once again, they'll be happy to charge you huge per-byte fees to get it.

Same goes with software, which can only be put on your phone after getting a special digital signature from Verizon for a fee. And that's only if you get the optional (and expensive) link cable. Do you spot a pattern?

They accomplish these feats by locking the customer down. Through partnerships with Motorola, Verizon phones are crippled to lock out users. Verizon says this is to prevent "fraud" (bluejacking) and tampering? To Verizon, this is "Digital Rights Management," your friend DRM. Any number of other scenarios are easy to think up in which media giants lock customers up to suck them dry. Verizon is the first to truly master it. They won't be the last.

Many say this is a prime example of how big media might use DRM. Selling songs isn't profitable enough. Force consumers to rent it instead. Songs no longer cost $1 to keep, they cost $1/month. This has never been possible before because media companies generally lose control over their content once it enters the consumer's hands. Not so with DRM. Such abuse can be very creative, as the Verizon example shows. So we do have something to fear from DRM after all.

It all comes down to the fact that DRM takes control away from the consumer and gives it to the content provider. In some ways this is good, as it prevents minor piracy. But when abused, it can be abused horribly. Consumers have always enjoyed a great deal of rights in this country, including some implicit ones like the ability to break open and do whatever they want (privately) with anything they buy. DRM changes this age old ability, locking consumers out.

We all know giving content providers too much control can lead to abuse. The Verizon case is the perfect example. Hopefully, consumers will be savvy enough to switch competitors and shun this sort of practice. One can hope.

The Short Road to Making Open Source Illegal

2005-01-21T03:36:00.000-05:00

California's Senate today introduced a bill that would make anyone a criminal who:

...sells, advertises, or distributes peer-to-peer file sharing software, as defined, that enables the user to electronically disseminate recordings or audiovisual works over the Internet who fails to exercise reasonable care in preventing use of the software to commit an unlawful act with respect to a commercial recording or audiovisual work, or a violation of provisions related to production,possession, distribution, or advertisement of obscene matter [...]

Notably, the bill defines Peer-to-Peer software primarily as a means to illegally distribute copyrighted goods, or illegal material (child pornography). While it is true that there are many, many P2P networks that traffic in such goods, defining P2P as it is here is simply shortsighted. P2P software can and routinely is used for legitimate means. It is a completely format agnostic technology. It works the same whether it is child porn or Ph.D research thesis papers being traded. Attempting to make the technology illegal will accomplish nothing. It will only have radical chilling effects on the industry.

"But how does this relate to OSS," you ask. Open Source Software is, obviously, open source. Anyone can download the software and do with it as they will. Developers of OSS programs have no control over what users do with their software. Laws such as this one that require some form of control over what users can and can't do with the software are not compatible with open source. It is not possible to block users from using a piece of software so generic as a P2P server for illegal means if the source is publicly available. Any restrictions placed in the source could easily be "fixed" to trade in illegal files. Considering the ease of bypassing the restrictions, it is unlikely "reasonable care" will be assumed.

This leaves only one reasonable choice for the developer of P2P software: close the source. In a back door way, laws like this one are bound to essentially make the practice of leaving source code publicly available legally quite risky. While developing the software is completely legal (see MGM v Grokster), making the source publicly available would be a crime, pure and simple.

Bills such as this one have been making rounds as of late on Capital Hill and in various state Senates. At all levels there has been heavy resistance to such new laws because of how they commonly misunderstand the technology involved and are extremely broad. Unfortunately, too many congressman and senators see the piracy issue as something so important that it must be stopped at nearly any cost. I can only hope the issue is resolved without trampling on the rights of OSS developers such as myself.

A virus is invading your TV

2004-12-17T01:59:00.000-05:00

Today's world, especially in America, advertisement has become a staple part of our media world. From popups to TV commercials, they're everywhere. Without them so many of our many forms of entertainment would vanish. Or would they? Growing louder and more obnoxious by the day, consumers hate them. But they're here to stay.

I saw the movie Blade Trinity about a week ago. Though the movie itself was rather boring, I noticed something subtle that caught my attention; or rather my disgust: advertising. After sitting through about 15 minutes of very loud commercials for American Express (among others), I was treated to another 10 minutes of movie previews. Finally into the film... Apple PCs that popping up in various scenes. A truck driving up prominently displaying a giant GMC logo reminiscent of a TV ad. And finally the 2 minute segment devoted entirely to how much the female lead loves her iPod. The first two are not surprising. Product placement is common in modern movie making. But what amounted to an iPod advert mid-movie was disturbing.

Our favorite female vampire slayer straps on her arrows and armor, but can't forget her iPod! The scene showed Jessica Biel selecting songs on iTunes to transfer over to her iPod, while another character explains how she always has to have her iPod on during hunts. This goes beyond product placement to blatant and distracting advertising. Such a display would break the illusion in any movie, much like if a character picked up a Coke proclaiming, "ahh, so refreshing. Try it today!" in the middle of a tear-jerking ending.

blurring this line between advertising and real content bothers me. For one it seems to encroach on the artistic nature of films, disrupting their illusion in a fantasy world. All this serves to diminish the value of these forms of entertainment, making them much less valuable.. Ventures are driven far more by cheap profits than by any artistic vision. The problem does not lie in the film industry lacking a profit stream to pay these movies: the $8+ ticket alone easily covers that, with plenty of profit. No, the product placement and overt ads are clearly for maximizing profit at any cost. The film industry knows most consumers will hate, but tolerate, the 25+ minutes of ads and previews before the movie, and will try to ignore the various cheesy product placements. It seems even this isn't enough to scare consumers away.

Such advances in the "less content, more ads" mindset are not limited to movies. TV ads are becoming longer and more frequent. Some primetime children's shows can have very long ads every 5 minutes or less, making the segment literally over half advertising. It is surprising that anyone actually watches TV anymore when they have to wade through so much advertising to get to the content they sat down to see. Movies on TV are also a culprit. My last attempt to watch Mission Impossible on TBS resulted in me giving up half-way through because very long ads would interrupt very serious moments in the movie, breaking its illusion and emotional impact.

At the start I alluded to the possibility of this wave of ever more intrusive advertising becoming obsolete. Is it possible? Maybe partially. One day when the patent-induced stranglehold on micro-payments expires, there may be another option. For TV, views would be able to pay a small fee, say $0.25 to watch CSI:Miami, and another $0.10 to see a Survivor rerun; all completely ad free and on-demand. I believe most consumers would easily shell out a few cents, maybe even a dollar, to see their favorite shows ad-free. Unfortunately for us, the absurd micro-payment patent won't expire until nearly 20 years from now.

Even with such possibilities, an ad-free future is very unlikely. Each time an ad-free venue pops up, advertisers find some way to sell their product. In movies, it become product placement. In Cable and satellite, we get ads despite our monthly fees. In TiVo, it will soon be pop-up ads that scroll by while you fast-forward past the TV ads. Even if a service is initially billed as "ad-free for a fee" it will eventually spawn more ads than ever. They will always find a way.

I suppose we have to accept our ad-ridden society. It's loud, demanding commercials and cheesy product placements are here to stay.