It’s about that time where I wrap the server side of this up and finish the TCP and named pipe implementations so that our client software can send queries from localhost for precompiled searches and from remote LAN addresses on the switch for TCP JIT requests.
That’s what I’m working on now. The other day I optimized a little bit, but today I’m getting rid of that and putting it into serialized structs for easy of use.
That way if the API changes, the structs can change and it will be as easy as that. No messing around in the bytestream!
God. I know I don’t think you exist, but please. I LEARNED MY LESSON.
Socialism isn’t cool. I repent. I repent.
Micheal Moore really is just a big fat loser.
My 7th grade vietnam vet social studies teacher was right. He was RIGHT GOD.
Please make it go away. I’ll do anything.
I was in the hall talking over search with a coworker. I forget that some people aren’t super up on what I’m doing.
So here goes. This is by no means the final version of search that I’m coding now. 255 * 8 bytes per line plus id and pagerank data(~200 byte per mem struct * a few billion) is not going to be acceptable for accuracy in a final search engine. I am only doing this because I only have access to quad core boards with about 4GB of ram TOTAL. For the kernel, the DMA and everything else. I only have about 2-3 gigs of RAM to use and I have billions of records to search from(indexes of the whole web).
So when we get to series B and we get the really expensive hardware with tens or hundreds of gigs of RAM, I will have to re-write this system including the memory manager and so forth to utilize the custom hardware, and to make the search accurate on the order of Google.
I only have access to low cost boards, cpus and hardware from TigerDirect.ca and other retailers. Even our Montreal supplier CiaraTech does not have the custom hardware necessary to build a real search engine. This is big time .edu stuff.
So with that said, this is kind of a search demo, but a very advanced one. Hopefully the demo will be impressive enough for series B funding next year so we can get that custom hardware. As far as getting it otherwise, that would be impossible, because it’s just too costly.
Yeah, reality sucks right?
Investors just want to see it working, so I can get it going that far. 1M for hardware is just a number to them and it may as well be custom toaster ovens we need for all they know or care.
UPDATE: This is probably the best I can do for the beta release. $2208.64
I’m going to use the board with this ram for powering the search memory manager. It’s not 50GB, but it will have to do. 2.2k is certainly more affordable than a million dollar system, but it’s going to be slower, because even with 8GB I won’t be able to load the full 8B page index. Not even at ~200bytes per record(including 21 crunched finite fields). Do the math.
UPDATE2: Actually I can probably get 2 sets of non-matched for under 2k. I am certainly looking to save money, so I will probably go that route.
UPDATE3: Just the tax money I send out this Thursday could buy 3 fscking sets of these @ 2k each. They will waste it all on crap. I will have to wait until next month to buy these. If BeerCo was in the US right now our search engine would be done with about 50 employees and it would already be selling SEO. Fsck this so fscking bad.
UPDATE4: I can’t wait to get out of here.
The VC and adviser I spoke of 2 posts ago has just gotten back to me and is interested in meeting. So I will probably go meet him after I go meet for the venture seed funding later this month. He is located in the bay area, so he will be able to help us locally while we set up shop in San Francisco. He is also affiliated with UC Berkley so that will probably help too if everything works out and we come to an agreement. I’m keeping my fingers crossed because he seems like a really great guy.
You can have the best code in the world, but there is more than just good code that makes up a successful venture. I am only really able to write code since I am a programmer. I am an OK manager, but I don’t think I am a very good corporate president. I would have liked to have been CTO instead had there been enough people on the board. Again there was a couple people on the board, but the BDC took care of that when they asked that all INVESTORS give up their personal financial information. As if owning a few shares in the company means that you have to release your bank records. That’s the way they do things here. That’s why I’m leaving. That pretty much destroyed BeerCo Canada about a year ago and the company has been in suspended animation since then.
The US company will be a great fresh start to do well in a low tax high demand environment.
Yup. Today I’m working on IPC and the TCP socket implementation on the searcher.
I’m actually doing this outside of Eclipse and without a makefile, so that’s interesting. I actually find that Eclipse has so much stuff it actually makes development slower on smaller projects. I just use the shell to compile. Sometimes I use custom makefiles. I have RMS’s book on GNU make, so I’m in good company.
I’m just now starting to get fees on remittances from 2006. These have come out of nowhere and they are 2 years late(I would dispute them since they have no basis, but that would only enrage them). This is on top of the huge amounts I’m regularly paying them.
I think my best course of action is to just pay whatever they want without disputing them and leave. It’s going to get out of hand if they are reading this blog and want revenge or whatever because of my comments.
I’ve never had so many “fans” in Montreal before…
I will go to a meeting in the US after xmas to discuss venture funding for the beta stage between the phase 1, the social search code named “spock” and phase 2, the full search engine code named “shangri la”.
While this is just seed venture funding, along with my own funding, it is still absolutely necessary to get the search through the private beta on phase 1 then phase 2, and into RTM.
As discussed all of the money in this venture will be put into a US company, not into BeerCo Canada. So it won’t be taxed at 60% before we can even use it for R&D whilst the American competitors pay about 20% tax with low fuel taxes and 6% sales tax. Then they sell the stuff they developed CHEAPLY IN THE US to us in Canada and drown our Canadian products that were developed at an insane tax rate. So with this venture the product will be on EVEN GROUND for the first time. For the first time I will not be at a great disadvantage due to govt of Canada against our US competitors.
I guess I won’t even be in Canada any more soon, so I don’t know why I’m complaining about this. I guess I am still bitter. I probably will be for the rest of my life.
So if this gets approved, the next step will be series B funding and that will probably take place when I have moved everything to San Francisco. I have been in contact with a VC and adviser, though I am not sure if I will end up working with him. Naeem if you are reading through the blog today, I would like to know if you are going to commit on this or not on the terms last discussed so get back to me on that.
You may think it weird that the adviser I conferred with is Indian. A lot of people think I am prejudice against Indians, but I’m not. I just don’t like Indian companies that misrepresent themselves. They often times call me at the office asking if I would like to outsource our contracts to India “like Microsoft and IBM”. Unfortunately they fail to mention that MS and IBM spend billions on foreign infrastructure and training. These companies have no training and like to fool people into thinking that outsourcing to independents in India is the same as spending millions for setting up a corporate office, flying everybody over from India to the US for intensive training, and getting them back there at a salary that is good enough to where they won’t steal from the company.
People don’t understand the difference when they hire Indian outsourcing companies, and they take full advantage of this lack of information. That is why I really dislike Indian companies. I don’t dislike Indian people as I judge every person based on merit.
I hate generalizing about India and China, but they have SUCH bad records, and somebody from those countries with a “virtual office” in the USA calls me every week or so trying to fraud me out.
UPDATE: The day the US funds Celine Dion and Victor Lucas or anybody of the genre, I am leaving there too. I am not prejudice against Canada. Govt of Canada did a horrible job. They’re crooks. Most of their programs would amount to communism except their true purpose is as an excuse to steal money. I would leave any country like that. I mean I am a Canadian citizen as well as a US citizen but I am powerless to do anything. In elections you get a ballot with like 2-3 people on it, only for your local MP, not for the president or the head of state. And the ballot is just big circles you have to X with a pencil. A PENCIL. It’s not scantron circles, they are like big round circles on a notepad type piece of paper and you have to give it back vote side up to the ballot takers. Why can’t we vote for the head of state of Canada? Because it’s the Queen of England. The Queen charters the pseudo type agencies like BDC, and we can’t say no. Even if we wanted to.
USA - /boot/kernel-usa
Australia - SELinux
Canada - ln -s /boot/kernel-usa /boot/kernel-canada_plus_$5000_tax
“PLUS a promise that the govt will stomp out all other local alternate kernel developers in exchange for exclusive ln -s rights”
“No really, we’re here to protect your R&D investment, we’ll stomp that out too at our SMBs.”
“My! That sure is a mightée big envelope, is that for me?”
“I wear nice suits, I’m a good person”
India - /var/spool/mqueue/spamtogo
China - /tmp/perl_telnet_trojan_upload
Japan - /usr/lib/libc.so
Russia - gcc moscow.c -o vodka
France - g++ moncul.cc -o bravo
Spain - ls -la
Argentina - rm -fr / && echo “hacked by team S1z3rZ”
Mexico - gdm (re: de Icaza)
What I’m doing now is optimizing so that queries for new searches can be passed with the least bytes possible as to increase speed across multi-billion record searches and mutli-million precompiled result caches.
Optimizing is something I like because it’s done after the main code is written, and it only requires slight modifications.