You don't need a permission slip to start a startup :)

Let me talk a little bit as a rare case: I've worked now under 7 advisors on 7 different projects and now I'm finally almost done. Most of the projects have been as you describe: "after almost a year, it is fair to say that I have good view of the project and it is almost surely a giant failure."

That's how most projects are. I'm not sure why, but I think it has to do with the fact that the projects are "planned research." The process of getting funding requires planning something that is intrinsically impossible to plan. You write a grant on hope, with the large picture in mind and then you get down to the details and things don't work out. This is normal.* Especially on the time frame of a new graduate student, it seems terrible.

To some extent, though, the system works. It does so for the same reason that some startups work: because as you look at the details you find new things that you couldn't have predicted. Those new things are your research. I'm working on a "failed project." But after 1.5 years working on it I am ready to begin writing a dissertation that I am proud of. Why? Because I found neat things along the way. That's how it works.

If you are in a good lab and surrounded by good people, I would recommend that you don't focus as much on the larger project as on learning all you can from the people around you and on understanding the details that your project will lead you to focus on. It is in helping other people, tracking down details, and playing with interesting questions that you will find the great science, not directly through the success of the larger project.

By the time I finish, I will have learned a lot. But I will have spent almost 8 years in graduate school, 2 for a masters in one field and almost 6 for a PhD in another field. If I had had my current perspective from the beginning I would have been a little more patient and I might thereby have saved myself three years. So there is a cost to hopping around even if there are benefits in terms of perspective.

Best wishes.

*I know of one exception. He explained his system: he does the research, then he gets grants for his research. This gives him time to do new research and he writes new grants based on that. He delivers because he only proposes to do what he has already more or less done!

Very inspiring modern design.

Grad school, for many people, seems inevitably to entail some prolonged dark night of the soul. Accept this as part of the process. It can cause you to lose yourself and your connection to what you really care about... or you can use it to strengthen these things. But you have to do that yourself. Don't expect anyone to help you. Models like "a community of teachers and learners" or "a thriving research environment" can lead to serious depression when they don't deliver. A better model is something like "an odyssey of solitude to test you".

I naively assumed that grad school would be an extrapolation of undergrad: I would be continuing my education by going deeply into a particular field. That was a big mistake. Eventually I realized it had nothing to do with education or even particularly with ideas. It was job training for becoming an academic. The real subject matter was: how to get an article published, how to fill out a grant application, how to give a talk at a conference. (Edit: oh, and how could I forget: how to pay as little attention to your teaching as possible.) As for learning the field, you were expected to know it all already and fill in any gaps on your own (we were handed a list of 500 books to read). I thought this was stupid and rebelled against the system in all kinds of ways that didn't help. Eventually I accepted that it is what it is, and turned my attention to what I wanted. Instead of changing the context, I figured out how to use it for my own purposes. It was hard to let go of my expectations - particularly to stop expecting mentorship or guidance from teachers. But doing so led to a lot of good things.

I divided my work into two categories: what I had to do to "feed the animal" (busywork and formal requirements) and what was meaningful to me. The strategy was to minimize the former (not eliminate it - don't fight the beast, petting works better) and maximize the latter. This worked well. Ironically, a lot of the praise and mentorship I had been craving started raining down on me once I no longer needed it.

Once one figures out how to feed the animal (in your case that sounds pretty easy), a graduate stipend can be a great basis for doing whatever you want. In my case, I used the last year of my fellowship to get back into programming. That was because I figured out I didn't want to stay in the field I was in; I had gotten what I needed out of it, and an academic job in that field would have been a waste. On the other hand, if I had had a passion for that field, I would have used my stipend to do work that I cared about, whether or not it was related to my official responsibilities. Not that I would have abrogated my responsibilities (that's the feeding the animal part) - just minimized them. If someone had asked me, "Why are you doing this other stuff" I would have said "That's my side project" or something like that. I also would have reached out to other people in the field, perhaps by sending them drafts of papers and so on. The point is that I would have done all this on my own and built up my own network of relationships.

This is a long comment that I could easily make longer! In any case, good luck.

Edit: ok, one more thing. It can be really useful to find a kindred spirit or two at your institution. They don't have to be in the same field. I would go exploring a little if I were you.

A year later, I put up my first > $500 software, not really expecting anything to happen. A week later, a sale came in, a few days later, the same.

There was no marketing plan, no competition analysis, no snazzy web designer or any of all that. Just put up the site, list it on download.com.

Software is an easy business. The hard part is in learning all that stuff so you can actually make the software (or know how to tell people what to do), afterwards it's pretty easy if you just release your stuff.

Congratulations! Your training is nearly complete! Welcome to the ranks of the enlightened! ;)

Your essay is pretty comprehensive and one can't really do it justice in less than an entire evening in a bar. But I'll offer some random observations.

You need to focus on a concrete goal. "Research is my main dream" is not a goal: It's a not even a real mission statement. "Get out of grad school with my degree" is a goal. "Get out of grad school right now, degree or no degree" is a goal. "Hang out in grad school while starting three YC companies" is a goal. "Go skiing every weekend until they kick me out of grad school, then get an industry job" is a goal. None of these is necessarily better than another; it depends on you.

You need to learn how a research career plays out in the real world. No, scratch that -- you've obviously learned it; you need to take a vacation from academia long enough to accept it. ;) The fact is: if the reviewers are happy, and the funding is good, you're an academic success. That's the goal of academic research, and if it disgusts you, you should get out of academia permanently. Because, frankly: "Finding elegant solutions to difficult problems that have wide-reaching implications" is almost uncorrelated to success in academia. You can be nearly as successful, with much less risk, by publishing over-complicated solutions to easy problems, or non-solutions to difficult problems, or incomprehensible solutions to niche problems. And you will find that the overwhelming majority of your colleagues spend most of their time on one of these paths, because they require less risk and less time, which leaves more time to go to conferences and write grant proposals and supervise students and curry favor with your colleagues and deliver lectures and all the other things that comprise actual day-to-day life in most academic jobs.

You need to realize that you are probably a very successful grad student. You can't tell right now, because you're still absorbing the truth: Most research projects are failures. That's what research is all about: Failing, over and over, but taking copious notes each time so that you have an idea of why you failed. As a student, such failure matters very little. My own Ph.D. thesis was a big catalog of various mistakes, ranging from small-scale implementation difficulties to grand-scale theoretical misconceptions that took three generations of grad students to unravel. And it went over just fine. People love reading about other people's learning experiences. It helps them learn what to avoid.

I'll repeat this, because it's important: You're a student. Nobody expects you to actually solve an earth-shattering problem. They expect you to do a bunch of work, help write some grants, write up something that your committee agrees is novel (but not necessarily earth-shattering) and then graduate and go work on something else. Which you should do. Unless you decide to just start working on something else right away, which would also be good.

Before this post gets any more Steve-Yeggesque: Yes, you can do research outside of academia. (You sound like a comp sci student, so take a moment to pity those EEs whose graduate work requires millions of dollars of capital equipment and a sizeable staff of techs. And even we can find ways to do research outside of academia.) You will find that it's much harder to get recognition for your work outside of the academy, but you have to ask: Are you in research for the adulation and the money and the girls, or are you in it for pleasure? Consider that many of the most successful discoveries were made by (what were then regarded as) semi-obscure cranks. (Think: Mendel.) Consider the delightful personal pleasures that crackpottery has to offer.

And, finally, I would note that if industry pays 30-40% more than your current job, that means you could work 60% of a full week at an industry job and still have time left over for your side projects.

That is brilliant! It reminds me of two things.

One is Craig Larman's work on the history of the waterfall process. Larman posed the question: given how broken the waterfall process is, how was it possible for any project to succeed? (The failures of course are easy to explain.) So he went back and interviewed people who were on famous waterfall projects that succeeded. And they told him: "Of course we knew it would never work to do all the design up front. So we built the system and then went back and wrote the spec, handed it in and got it signed off, and later submitted the actual code."

The other thing it reminds me of is Andrew Wiles writing a stack of papers in advance so he would have ten years to work on Fermat's Last Theorem while still publishing regularly.