Monday, March 31, 2008

Getting started on search

Spent an hour tonight playing with Lucene.Net. I've got a real simple command line version of search going. I also think I came up with a good idea for queueing indexes. Hopefully this weekend I'll have time to hack on that. For now search seems like an easier problem than I expected though.

Here's my real simple indexer and searcher:



using System;
using System.Configuration;
using System.Collections.Generic;

using Lucene.Net.Index;
using Lucene.Net.Documents;
using Lucene.Net.Analysis.Standard;

using MySql.Data.MySqlClient;


namespace Grurrah.Tools {

public class Indexer {

private static readonly int GrurrahIdColumn = 0;
private static readonly int GrurrahNameColumn = 1;
private static readonly int EntryTextColumn = 2;
private static readonly int TagNameColumn = 3;
private static readonly int RatingAvgColumn = 4;

private static readonly string QueryString = "SELECT Grurrahs.id, Grurrahs.name, Entries.text, Tags.name, AVG(user_grurrah_ratings.rating) FROM Grurrahs " +
"RIGHT JOIN Entries ON Grurrahs.id = Entries.grurrah_id AND Grurrahs.entry_index = Entries.entry_index " +
"LEFT JOIN tag_map ON Grurrahs.id = tag_map.grurrah_id LEFT JOIN Tags on tag_map.tag_id = Tags.id " +
"LEFT JOIN user_grurrah_ratings ON Grurrahs.id = user_grurrah_ratings.grurrah_id GROUP BY Grurrahs.id, Tags.name";


private static MySqlDataReader reader;

public static void Main ()
{
LoadData ();
IndexData ();
}

private static void LoadData ()
{
ConnectionStringSettings cs = ConfigurationManager.ConnectionStrings ["Read"];
MySqlConnection con = new MySqlConnection (cs.ConnectionString);

con.Open();

MySqlCommand cmd = new MySqlCommand (QueryString, con);
reader = cmd.ExecuteReader ();
}

private static void IndexData ()
{
IndexWriter writer = new IndexWriter ("Indexes", new StandardAnalyzer (), true);

while (reader.Read ()) {
if (reader [GrurrahNameColumn] == DBNull.Value)
continue;
writer.AddDocument (IndexGrurrah ());
}

writer.Optimize ();
writer.Close ();
}

public static Document IndexGrurrah ()
{
Document doc = new Document ();

if (reader [RatingAvgColumn] != DBNull.Value)
doc.SetBoost (reader.GetInt32 (RatingAvgColumn));

doc.Add (new Field ("name", reader.GetString ("name"), Field.Store.YES, Field.Index.TOKENIZED));
doc.Add (new Field ("text", reader.GetString ("text"), Field.Store.YES, Field.Index.TOKENIZED));

return doc;
}
}
}




using System;
using Lucene.Net.Search;
using Lucene.Net.Documents;
using Lucene.Net.QueryParsers;
using Lucene.Net.Analysis.Standard;



namespace Grurrah.Tools {

public class Search {

public static void Main (string [] args)
{
if (args.Length < 1) {
Console.Error.WriteLine ("You must supply a query parameter.");
return;
}

IndexSearcher searcher = new IndexSearcher ("Indexes/");
QueryParser parser = new QueryParser ("text", new StandardAnalyzer ());
Query query = parser.Parse (args [0]);
Hits hits = searcher.Search (query);

for (int i = 0; i < hits.Length (); i++) {
Document doc = hits.Doc (i);
Console.WriteLine ("{0}: {1} -- {2}", doc.Get ("name"), doc.Get ("text"), hits.Score (i));
}

searcher.Close ();
}
}
}

It's official, Grurrah.com is open source

Just checked the first source code drop into svn.

Sunday, March 30, 2008

New WYSIWYG Editor

Tonight I worked on a new WYSIWYG editor for Grurrah. It was a pretty simple task since I am using the gaia RichEdit control and the MultiView. I wish I had known about MultiView before, since I've basically done a manual version of the multi view all over the place in Grurrah. I'll clean that up sometime soon, removing a bunch of manual setting of controls visibility and using MultiView.



The hardest part of making the editor was finding icons. I'd like to add list support to the editor, but tango doesn't have list icons. I will also be adding image support soon, I just need to think through the details of image upload first.

I also want to add support for linking to other Grurrahs, but I'll need an icon for that. I'm planning on using the gaia:AutoComplete control for the link to other Grurrahs window, filling it in with the most popular Grurrahs, and maybe eventually I'll fill it with related Grurrahs.

Developing and Deploying Grurrah

Grurrah.com is run on a single linode virtual server. The virtual server is running open suse 10.3. I am using apache2, mod_mono, mysql (with the MySql .net connector), and memcachedto actually run the site. I am also using a couple of open source libraries. For all the AJAX support I am using Gaia Ajax Widgets, for authentication I am using a MySqlMembershipProvider and MysqlRoleProvider, and I am also experimenting with the UrlRewriterNet.

I haven't deployed the UrlRewriting stuff yet because it required that I add a missing property to Mono's System.Web, and it's a bit of a deployment hassle. I think it's best to forget about url rewriting for awhile, since it's side effects will slow down development. Url rewriting is a feature I have been planning from the beginning though, I think one of the best things about wikipedia is that I can just type what I want right in the firefox address bar, and go where I expect. I plan on something like grurrah.com/grurrahs/Compost working in the future.

With the exception of the one little issue with UrlRewriterNet, I've been able to download, compile, develop and test on Windows and deploy straight to linux/mono without any problems. Normally I don't develop on windows, but for this project it just made sense to use the Visual Web Developer. Once things are a little further along and more of the development effort is focused on backend stuff, I hope I'll be able to go back to developing on Linux, since it's much more familiar to me.

I remember when I first started working on Mono, one of our main goals was to make development on Windows using third party libraries and deploying to Linux possible. Five years later, it's not only possible, it's easy.

Saturday, March 29, 2008

A very soft launch

My goal was to launch Grurrah.com and announce it's presence to the world during earth hour. It would have been a beautiful moment, sitting in the dark, conserving energy while uploading my website that will hopefully teach people how to conserve energy.

However, I ran into a bit of a speed bump with url redirecting. It was one of those deals where you spend hours trying to fix something, running into and fixing issue after issue until finally you hit a show stopper and realize that there is no way you can get around it.

Luckily, it's earth hour somewhere. In fact, I managed to launch during west coast earth hour, and all my lights were off.

So Grurrah.com is now up. I've decided to not announce it too heavily for a few more days, since there are still some issues to iron out and I am not sure how well my linode can handle the load. So no big press release, no blog entry on monologue, not even announcing the site on facebook yet. My soft launch consisted of one SMS conversation with my sister that went something like this:

jackson: are you near a computer?
lindsay: i am at dinner
jackson: checkout grurrah.com, it's a new website i just launched

Hopefully the server can withstand the load.

Wednesday, March 26, 2008

Hot? Or Not?



Tonight I wrote a little rating control using the gaia ImageButton. Pretty simple to do the client side stuff, for some reason it took me awhile to get the SQL right. My SQL definitely needs some serious optimization work but everything is working fine for now.

Sunday, March 23, 2008

Screenshots

The Main Page:


The Grurrah Page:

Trimming

Today I decided I just have to trim back some features if I want to move forward at all. I have had very little time to hack on Grurrah lately and the little time I have had, I feel like I have just been tinkering with my discussion control, trying to make it fit into the main Grurrah page. This is difficult for two reasons 1) The main page isn't even done and 2) I am a horrible designer.

So for now I have decided to drop my discussion control. It's too bad because it's my neatest control, but in the interest of getting the site up and running. I am going to go discussion-less for awhile, and have that be one of the first features added once the site is up.

I purchased my linode.com space today. Not sure how long it will take before I can access my node. I am hoping that it's 48 hours max. I really really really want to have _something_ posted by Wednesday morning.

Stay tuned, the home page is looking pretty nice and I will be posting a screenshot later today. HAH! Stay tuned, as if anyone is reading this blog but me.