Mm, okay, so I finally got around to adding unit tests to my project. I was right about ReSharper; it rocks. The license is good, too. They call it a "named license", where the person named can be running one instance at a time, anywhere. So, you can install it on several machines, and as long as you are only using one of them at a time, you are adhering to the license.
I like it.
I had to make some decisions regarding the unit testing. Spring.NET has some built-in NUnit thing, but I chose to go with the latest and greatest, so I'd know what I was getting. I don't like to look for old docs.
As usual, adding test meant re-working a bunch of the code to accommodate it. The code is better for it, of course, and I am once again reminded why you add tests from the beginning. I won't learn, though.
There were special considerations where Spring.NET is concerned. Should I use it to wire everything together for the tests, too? It would be appropriate, at least at an integration level. Did that make sense at the unit level?
I decided, after some experimentation, that no, that "manually" wiring things together in the test suites was the way to go, with some separate tests including Spring.NET to be sure that part is working okay.
The configuration flexibility that IoC brings is inappropriate for testing, I think, because the tests should always reflect what needs to be tested, in isolation. External factors, such as what is in a config file, shouldn't be part of the tests.
Many of the changes I had to make in response to this decision dealt with writing getters and setters for private variables. Spring.net may be able to set private variables, but I'm not going to jump through the hoops to do it from a test. Once I was forced to make a decision, going with "anything that can be configured should be configurable to whatever creates it" strategy, and adding some extra checks so things don't get broken seems best, at least for now.
I'm sure I'll get pissed off about it at some point.
Friday, April 24, 2009
Versioning, and Versioning on Windows
I'm currently using git for my versioning needs, specifically msysgit on Vista. I got spoiled on distributed version control systems when I was running Ubuntu on my desktop. Bazaar was my choice then, especially since it ran just fine on Windows, too.
I've moved onto git since then, mostly because it looks like it has the most momentum. Now that I've made the switch, I'm happy. It is nice to finally have a tool which fits whatever workflow you want to throw at it, instead of having to adjust your workflow for it.
I've been a proponent of VCS as an IT goon for a long time. They aren't just useful for versioning code, they are useful for versioning all kinds of files. Some of the easiest deployment environments I've worked in and built had their production content under version control. Rollbacks were trivial, the "what changed" question was easily answered, revision numbers were part of change management.
Making a copy of everything isn't versioning, it is only rollback. Copies are just backup. It is good enough, but it isn't particularly smart.
Distributed version control is even cooler than the old fashioned (like, a couple of years ago!) centralized VCS. The previous generation required a centralized server, even if it was one user on the same machine. DVCS lets you start versioning on-the-fly.
Having things versioned makes it easy to fix your mistakes, giving you the freedom to experiment. When you screw up, you just go back to where you started. Or you can see exactly what you changed that might've screwed things up, so you can fix it faster. Are you building up a text document describing whatever it is you're working on, even just as notes? Version it. Seriously.
Of course, on Windows this all sucks. There is a GUI for git, and it is adequate, but it doesn't really unleash the Power of Git. The tools are good enough that there isn't really a reason for not using them, though.
Microsoft's versioning story is very weak. Team Foundation Server ain't gonna cut it. We need it a little bit more integrated. And free. I mean, really, when people are happily giving away the next-gen technology, you can count that part of the product as "obsolete". I'm not holding out for something at the file system level. Yet. (nor network aware, yet, either - but if they want to be better, they've got to get ahead of this)
So, in the meantime, I have a couple of Powershell aliases to help me out. I want to spend more time writing better ones, but to really get into it, I'd need to write it as a cmdlet, and I don't have the time. And I shouldn't have to - something like this should be shipped in the OS. It should be pretty, it should be easy, it should be powerful, and it should be programmable. That's just to catch up.
Here's what I'm using in $profile (I've got gvim installed for my editor. Suck it, notepad):
I know, not much to go with. More complex aliases are prohibitive, and functions look like functions, parenthesis and all. As I get a better understanding of how best to use git, I'm coming up with what it will take to simplify my own specific workflow. I'll share them as I do them.
I've moved onto git since then, mostly because it looks like it has the most momentum. Now that I've made the switch, I'm happy. It is nice to finally have a tool which fits whatever workflow you want to throw at it, instead of having to adjust your workflow for it.
I've been a proponent of VCS as an IT goon for a long time. They aren't just useful for versioning code, they are useful for versioning all kinds of files. Some of the easiest deployment environments I've worked in and built had their production content under version control. Rollbacks were trivial, the "what changed" question was easily answered, revision numbers were part of change management.
Making a copy of everything isn't versioning, it is only rollback. Copies are just backup. It is good enough, but it isn't particularly smart.
Distributed version control is even cooler than the old fashioned (like, a couple of years ago!) centralized VCS. The previous generation required a centralized server, even if it was one user on the same machine. DVCS lets you start versioning on-the-fly.
Having things versioned makes it easy to fix your mistakes, giving you the freedom to experiment. When you screw up, you just go back to where you started. Or you can see exactly what you changed that might've screwed things up, so you can fix it faster. Are you building up a text document describing whatever it is you're working on, even just as notes? Version it. Seriously.
Of course, on Windows this all sucks. There is a GUI for git, and it is adequate, but it doesn't really unleash the Power of Git. The tools are good enough that there isn't really a reason for not using them, though.
Microsoft's versioning story is very weak. Team Foundation Server ain't gonna cut it. We need it a little bit more integrated. And free. I mean, really, when people are happily giving away the next-gen technology, you can count that part of the product as "obsolete". I'm not holding out for something at the file system level. Yet. (nor network aware, yet, either - but if they want to be better, they've got to get ahead of this)
So, in the meantime, I have a couple of Powershell aliases to help me out. I want to spend more time writing better ones, but to really get into it, I'd need to write it as a cmdlet, and I don't have the time. And I shouldn't have to - something like this should be shipped in the OS. It should be pretty, it should be easy, it should be powerful, and it should be programmable. That's just to catch up.
Here's what I'm using in $profile (I've got gvim installed for my editor. Suck it, notepad):
$vimexe = $env:USERPROFILE + '\sys\vim72\gvim.exe'
new-alias -name vi -value $vimexe
$gitexe = $env:USERPROFILE + '\sys\git\bin\git.exe'
new-alias -Name git -Value $gitexe
# git wants a HOME
$env:HOME = $env:USERPROFILE
$env:EDITOR=$vimexe
I know, not much to go with. More complex aliases are prohibitive, and functions look like functions, parenthesis and all. As I get a better understanding of how best to use git, I'm coming up with what it will take to simplify my own specific workflow. I'll share them as I do them.
Monday, April 20, 2009
MS or OSS? Integration v. Features
I have to make a decision about a testing framework for my little project. I can't put it off any longer, or I'll really regret it.
At least in this category there's no shortage of options.
The power of a well-integrated solution shouldn't be dismissed. There's real merit in being able to get things going with a minimum of fuss, at the cost of long-term flexibility.
In this case, it doesn't look like I'll have to worry about it. ReSharper apparently integrates NUnit nicely.
At least in this category there's no shortage of options.
- NUnit. Part of the xUnit family. Among them, they have very similar syntax and capabilities.
- MSTest. This has a similar syntax, and is really well integrated into Visual Studio
The power of a well-integrated solution shouldn't be dismissed. There's real merit in being able to get things going with a minimum of fuss, at the cost of long-term flexibility.
In this case, it doesn't look like I'll have to worry about it. ReSharper apparently integrates NUnit nicely.
log4net and Spring.Net, once again
Here's the goo to configure log4net in your spring.net config, assuming a property of type ILog named "log". It will initialize it with a named logger for "Document".
Not that much to it, after all.
<property name="log">
<object type="log4net.LogManager,log4net" method="GetLogger">
<constructor-arg value="Document">
</constructor-arg></object>
</property>
Not that much to it, after all.
Logging Should Be Added Early
The problem with starting a project just to see what happens is that you always forget the important things, because they "slow things down".
Like logging. When I was just poking around with a couple of components, wiring them together crudely, it didn't matter. If I wanted to see what the program was doing, I'd just break in with the debugger. Nothing to it.
Of course, now that I'm actually trying to get things to work well, logging is suddenly important. The only thing that is saving my ass is that I'm rewriting large chunks to better suit my scenario, so I can add logging at that point.
The only thing dumber than putting off logging is putting off a test suite!
Aw, hell...
Like logging. When I was just poking around with a couple of components, wiring them together crudely, it didn't matter. If I wanted to see what the program was doing, I'd just break in with the debugger. Nothing to it.
Of course, now that I'm actually trying to get things to work well, logging is suddenly important. The only thing that is saving my ass is that I'm rewriting large chunks to better suit my scenario, so I can add logging at that point.
The only thing dumber than putting off logging is putting off a test suite!
Aw, hell...
Saturday, April 18, 2009
Game Server Licencing?
I was just reading about the release of Demigod, from Stardock. Stardock has found a rare warm spot in my cold, cynical heart with their stance towards copy-protection: they don't use it.
Update: Since I initially posted this, they have moved up the game charts. They are showing good sales despite all the piracy.
Their last game, Sins of a Solar Empire, did quite well without copy protection, and it was hoped that this release would work out the same. I liked the game a lot - it isn't just some second-rate game desperate for an audience. It was good.
Well, not quite. It turns out, when the game starts up, it makes a request from Stardock's game servers. Or something like that. In any event, even people who copied the game without buying it were connecting to the multiplayer game servers. So, even legit customers were having a poor experience.
Some of the numbers that came out of this are interesting, too. There were about 120,000 connections, but only 18,000 of those were paid for. A lot of people are pointing to this as a failure, but I look at it as rather successful. That's 120K people who want to play the game. If they can't monetize that audience, then part of the failure is theirs.
Given the requirements of game servers these days, I certainly understand the desire of game developers to find ways to recoup the costs. How about selling licenses to those game servers? Heck, let me allow pirated copies to connect, and I'll pay more for the privilege. Stardock could be the official, sanctioned site, offering a higher quality experience. I'll help convert some of those freeloaders into actual income off of whatever crap I can scam from a local colo.
Maybe I'm just another spoiled internet brat, wanting more now that I've been given whatever it was I was whining about before. I do think game makers are missing the point when they take on the whole D&D GM-ish role. Running a game server is similar to running a role-playing game.
There's a lot of people who like to be GMs, and a lot of people who would prefer to play with those GMs. Game servers scratch the same itch, and there's money on the table for whoever satisfies it. People want to play, they want to play the way they want to play, and no single source can satisfy all the variations.
C'mon. I've always wanted to run my own WoW server. How hard could it be? ;)
Update: Since I initially posted this, they have moved up the game charts. They are showing good sales despite all the piracy.
Their last game, Sins of a Solar Empire, did quite well without copy protection, and it was hoped that this release would work out the same. I liked the game a lot - it isn't just some second-rate game desperate for an audience. It was good.
Well, not quite. It turns out, when the game starts up, it makes a request from Stardock's game servers. Or something like that. In any event, even people who copied the game without buying it were connecting to the multiplayer game servers. So, even legit customers were having a poor experience.
Some of the numbers that came out of this are interesting, too. There were about 120,000 connections, but only 18,000 of those were paid for. A lot of people are pointing to this as a failure, but I look at it as rather successful. That's 120K people who want to play the game. If they can't monetize that audience, then part of the failure is theirs.
Given the requirements of game servers these days, I certainly understand the desire of game developers to find ways to recoup the costs. How about selling licenses to those game servers? Heck, let me allow pirated copies to connect, and I'll pay more for the privilege. Stardock could be the official, sanctioned site, offering a higher quality experience. I'll help convert some of those freeloaders into actual income off of whatever crap I can scam from a local colo.
Maybe I'm just another spoiled internet brat, wanting more now that I've been given whatever it was I was whining about before. I do think game makers are missing the point when they take on the whole D&D GM-ish role. Running a game server is similar to running a role-playing game.
There's a lot of people who like to be GMs, and a lot of people who would prefer to play with those GMs. Game servers scratch the same itch, and there's money on the table for whoever satisfies it. People want to play, they want to play the way they want to play, and no single source can satisfy all the variations.
C'mon. I've always wanted to run my own WoW server. How hard could it be? ;)
What's With All The Code?
As an IT Goon, I spend a lot of time working with code. While I personally enjoy it, I think it is important for anyone whose career is so dependent on computers to know to be comfortable writing code on a regular basis.
The idea of writing a short script to automate a common task should one of the first. Computers are great at doing the same thing over and over again. Personally, I don't like doing that, so I tend to reach for the text editor pretty quick. Being lazy has furthered my career more than any certification.
There's more to it than that, though. Ultimately, any application is not only the code that gets installed, it is the underlying environment as well. The OS version, the common libraries, installed libraries, and settings for all of that (IO, networking, security, et. al.) Not to mention dependencies among the applications themselves, where certain applications all need to be installed in the same OS instance.
Various containers have mitigated a lot of the problems, but these, too, have to be administered. The more you know about the overall application (app, OS, and their interaction), the easier this is. I'm frequently amazed at how much money is spent on pre-packaged solutions to solve simple problems. It usually gets sold as "it solves this problem, and so many more!" Except that we don't have those problems, and many of those have simple solutions.
Really, in many ways developers are customers. This doesn't mean they're always right, but it does mean that IT is there to serve them. The more we understand about these customers, the better service we can provide. In the IT role, this means not only understanding how to write code, but what is involved in writing code. At the same time, we build our own toolset to make our jobs easier.
The idea of writing a short script to automate a common task should one of the first. Computers are great at doing the same thing over and over again. Personally, I don't like doing that, so I tend to reach for the text editor pretty quick. Being lazy has furthered my career more than any certification.
There's more to it than that, though. Ultimately, any application is not only the code that gets installed, it is the underlying environment as well. The OS version, the common libraries, installed libraries, and settings for all of that (IO, networking, security, et. al.) Not to mention dependencies among the applications themselves, where certain applications all need to be installed in the same OS instance.
Various containers have mitigated a lot of the problems, but these, too, have to be administered. The more you know about the overall application (app, OS, and their interaction), the easier this is. I'm frequently amazed at how much money is spent on pre-packaged solutions to solve simple problems. It usually gets sold as "it solves this problem, and so many more!" Except that we don't have those problems, and many of those have simple solutions.
Really, in many ways developers are customers. This doesn't mean they're always right, but it does mean that IT is there to serve them. The more we understand about these customers, the better service we can provide. In the IT role, this means not only understanding how to write code, but what is involved in writing code. At the same time, we build our own toolset to make our jobs easier.
Friday, April 17, 2009
Logging in Spring.NET with log4net, initialization
It didn't take long to run into a snag.
The docs I've found so far say to get your log thusly:
Being the lazy bastard that I am, I started copying that into various constructors. The thought occurred to me that I should be able to put all of this into the spring config, but then there's that whole "being the lazy bastard that I am".
It didn't take long to run into an unhelpful exception:
What had I changed? I hadn't been as aggressive committing as I should've been, so there were a lot of little changes to poke through.
In the constructor of FileCrawlerEngine, I used the log that I had just created:
Removing the Trace from the constructor resolved it.
Hrm.
I switched everything over to use log4net, directly, and it all worked okay. Even with a few threads thrown in. It takes even less upfront work. Create the log4net section, and follow the log4net docs. Just make sure that somewhere in your initialization, you call the following:
Then, use it in your code, thusly:
Using it directly exposes a better API, and it works as expected. I haven't gotten any errors, but I'll bet they are clearer than the one thrown by Spring.NET. It seems like Common.Logging requires more work for less results - I think I'll be sticking with the direct use of log4net, now.
It should be possible to springify the log initialization. That'll have to be a digression for later - I still have a lot of logging retro-fitting to do.
UPDATE: I took one look at adding logging to the SolrSharp library, and decided that now was as good a time as any to add the logging config to spring. I ran into a similar problem - logging in the constructor failed. It had a better error message, though:
Okay, so the property doesn't get set until after construction. That makes sense, at least for my problem. Too bad the Spring.NET guys didn't find a way around this for Common.Logging (and that I'm too dumb to figure it out myself).
UPDATE 2: Spring.net will call a specified initialization method, so that you can use the objects which are initialized by the container.
Reading the manual is for the weak. So, I'm weak.
The docs I've found so far say to get your log thusly:
ILog log = LogManager.GetLogger( "Outermost" );
Being the lazy bastard that I am, I started copying that into various constructors. The thought occurred to me that I should be able to put all of this into the spring config, but then there's that whole "being the lazy bastard that I am".
It didn't take long to run into an unhelpful exception:
Cannot instantiate Type [FileCrawler.FileCrawlerEngine] using ctor [Void .ctor(System.String)] : 'Index (zero based) must be greater than or equal to zero and less than the size of the argument list.'
What had I changed? I hadn't been as aggressive committing as I should've been, so there were a lot of little changes to poke through.
In the constructor of FileCrawlerEngine, I used the log that I had just created:
log = LogManager.GetLogger( "FileCrawler" );
log.Trace( String.Format( "Initializing FileCrawler for basepath {0}" ) );
Removing the Trace from the constructor resolved it.
Hrm.
I switched everything over to use log4net, directly, and it all worked okay. Even with a few threads thrown in. It takes even less upfront work. Create the log4net section, and follow the log4net docs. Just make sure that somewhere in your initialization, you call the following:
log4net.Config.XmlConfigurator.Configure();
Then, use it in your code, thusly:
log = log4net.LogManager.GetLogger( "NdxDocument" );
log.Info( "Informative" );
Using it directly exposes a better API, and it works as expected. I haven't gotten any errors, but I'll bet they are clearer than the one thrown by Spring.NET. It seems like Common.Logging requires more work for less results - I think I'll be sticking with the direct use of log4net, now.
It should be possible to springify the log initialization. That'll have to be a digression for later - I still have a lot of logging retro-fitting to do.
UPDATE: I took one look at adding logging to the SolrSharp library, and decided that now was as good a time as any to add the logging config to spring. I ran into a similar problem - logging in the constructor failed. It had a better error message, though:
"Error creating object with name 'Document' defined in 'config [C:\\Users\\michael.THREETREES\\Documents\\Visual Studio 2008\\Projects\\Crawler\\FileCrawler\\bin\\Debug\\FileCrawler.vshost.exe.Config#spring/objects] line 1' : Initialization of object failed : Cannot instantiate Type [NdxCore.NdxDocument] using ctor [Void .ctor()] : 'Object reference not set to an instance of an object.'"
Okay, so the property doesn't get set until after construction. That makes sense, at least for my problem. Too bad the Spring.NET guys didn't find a way around this for Common.Logging (and that I'm too dumb to figure it out myself).
UPDATE 2: Spring.net will call a specified initialization method, so that you can use the objects which are initialized by the container.
Reading the manual is for the weak. So, I'm weak.
Logging in Spring.NET with log4net
Wherein real progress is made...
I've mentioned that Spring.NET is a large project, with more than just the IoC container that I was babbling about. It also has log4net built in. At first blush, this seemed like a good thing. Then I got to the documentation.
The docs say use log4net syntax....
But they are working on a Common Logging...
Which they got going in 2006...
Okay, at least there's a "Common.Logging.dll" in the spring folder...
Hmmm, well the example that guy gives doesn't help much, where's the better docs? The example uses an external file - what if I don't want to do that? Is that INLINE?
Another chunk of doc sends us to the Common Infrastructure Libraries for .NET site...
The sum of whose libraries is logging. Heh.
Hm. Okee dokee. Does Common.Logging.dll include log4net?
Only one way to find out, I guess...
Added it's section name (the example from up there, and the corresponding configSections entry, to the app.config...nuthin. I'd expect to see something from Spring, at least.
Changed the logging level to "DEBUG" for Spring...still nuthin.
Okay, maybe some code is in order. How do I initialize this? Do I use the log4net syntax?
Back to the...oh crap...*sigh*...documentation.
Aha! Totally different. Yea, I could've started here, huh? But I didn't.
There's a few different adapters supported, including NLog, and EntLib. BTW, Microsoft's Enterprise Library is pretty nice - it has a lot of the functionality of Spring/Spring.NET, but isn't as complete. If you like a lot of what I'm talking about, but want an MS alternative, that would be it.
In fact, it might be more correct to re-title this post as "Logging in Spring.NET with log4net".
Log4NetLoggerFactoryAdapter seems to be what I'm looking for...
There's a lot of references to calling XmlConfigurator.Configure(). Is this from Spring.net?
Apparently, in the common/logging/factoryAdapter node, the configType of INLINE means "do it like you were using log4net, only you need this whole other section, too".
In goes a log4net section, then.
BLAMMO! no worky.
Okay, that ain't there. It's hiding in the spring.net distribution under lib\net\2.0, along with some other dlls I'll probably want some day. I need to get the source code and add it to the project, but this should do for now.
Added Common.Logging.Log4Net and log4net as references in the project, and that got rid of the errors, and I got a chunk of output. Cool.
To summarize (don't you wish I had already?):
Using INLINE in the app config tells Common Logging to call XmlConfigurator.Configure(), which is (apparently - I forget how to use it) log4net initializer. To get it all working, you have to create a "common" section, as well as a "log4net" section. If I understand the history of all this correctly, it was to simplify dealing with multiple versions of log4net, with the bonus of being able to plug in other logging providers.
I'm thinking it might be simpler just to use log4net directly. I don't anticipate having that particular problem. I'll try it out, first, though.
To get it to work:
Reference log4net, Common.Logging, and Common.Logging.Log4Net in your project.
Then update your app.config to look something like this:
Great - it's logging all of Spring.net's stuff. How do developers use it? Easy, once you have the references.
Too simple, once it's set up.
I've mentioned that Spring.NET is a large project, with more than just the IoC container that I was babbling about. It also has log4net built in. At first blush, this seemed like a good thing. Then I got to the documentation.
The docs say use log4net syntax....
But they are working on a Common Logging...
Which they got going in 2006...
Okay, at least there's a "Common.Logging.dll" in the spring folder...
Hmmm, well the example that guy gives doesn't help much, where's the better docs? The example uses an external file - what if I don't want to do that? Is that INLINE?
Another chunk of doc sends us to the Common Infrastructure Libraries for .NET site...
The sum of whose libraries is logging. Heh.
Hm. Okee dokee. Does Common.Logging.dll include log4net?
Only one way to find out, I guess...
Added it's section name (the example from up there, and the corresponding configSections entry, to the app.config...nuthin. I'd expect to see something from Spring, at least.
Changed the logging level to "DEBUG" for Spring...still nuthin.
Okay, maybe some code is in order. How do I initialize this? Do I use the log4net syntax?
Back to the...oh crap...*sigh*...documentation.
Aha! Totally different. Yea, I could've started here, huh? But I didn't.
There's a few different adapters supported, including NLog, and EntLib. BTW, Microsoft's Enterprise Library is pretty nice - it has a lot of the functionality of Spring/Spring.NET, but isn't as complete. If you like a lot of what I'm talking about, but want an MS alternative, that would be it.
In fact, it might be more correct to re-title this post as "Logging in Spring.NET with log4net".
Log4NetLoggerFactoryAdapter seems to be what I'm looking for...
There's a lot of references to calling XmlConfigurator.Configure(). Is this from Spring.net?
Apparently, in the common/logging/factoryAdapter node, the configType of INLINE means "do it like you were using log4net, only you need this whole other section, too".
In goes a log4net section, then.
BLAMMO! no worky.
{"Could not load file or assembly 'Common.Logging.Log4Net' or one of its dependencies. The system cannot find the file specified.":"Common.Logging.Log4Net"}Okay, that ain't there. It's hiding in the spring.net distribution under lib\net\2.0, along with some other dlls I'll probably want some day. I need to get the source code and add it to the project, but this should do for now.
Added Common.Logging.Log4Net and log4net as references in the project, and that got rid of the errors, and I got a chunk of output. Cool.
To summarize (don't you wish I had already?):
Using INLINE in the app config tells Common Logging to call XmlConfigurator.Configure(), which is (apparently - I forget how to use it) log4net initializer. To get it all working, you have to create a "common" section, as well as a "log4net" section. If I understand the history of all this correctly, it was to simplify dealing with multiple versions of log4net, with the bonus of being able to plug in other logging providers.
I'm thinking it might be simpler just to use log4net directly. I don't anticipate having that particular problem. I'll try it out, first, though.
To get it to work:
Reference log4net, Common.Logging, and Common.Logging.Log4Net in your project.
Then update your app.config to look something like this:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="solr" type="org.apache.solr.SolrSharp.Configuration.SolrConfigurationSection, SolrSharp" allowLocation="true" allowDefinition="Everywhere" />
<sectionGroup name="spring">
<section name="context" type="Spring.Context.Support.ContextHandler, Spring.Core"/>
<section name="objects" type="Spring.Context.Support.DefaultSectionHandler, Spring.Core" />
</sectionGroup>
<sectionGroup name="common">
<section name="logging" type="Common.Logging.ConfigurationSectionHandler, Common.Logging" />
</sectionGroup>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net"/>
</configSections>
<log4net>
<appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline" />
</layout>
</appender>
<!-- Set default logging level to DEBUG -->
<root>
<level value="DEBUG" />
<appender-ref ref="ConsoleAppender" />
</root>
<!-- Set logging for Spring to INFO. Logger names in Spring correspond to the namespace -->
<logger name="Spring">
<level value="DEBUG" />
</logger>
</log4net>
<common>
<logging>
<factoryAdapter type="Common.Logging.Log4Net.Log4NetLoggerFactoryAdapter,Common.Logging.Log4Net">
<!-- choices are INLINE, FILE, FILE-WATCH, EXTERNAL-->
<!-- otherwise BasicConfigurer.Configure is used -->
<!-- log4net configuration file is specified with key configFile-->
<arg key="configType" value="INLINE" />
</factoryAdapter>
</logging>
</common>
<spring>
<context>
<resource uri="config://spring/objects"/>
</context>
<objects>
</objects>
</spring>
</configuration>
Great - it's logging all of Spring.net's stuff. How do developers use it? Easy, once you have the references.
using Common.Logging;
...
ILog log = LogManager.GetLogger( "somelog" );
log.Error( "this is not an error!" );
Too simple, once it's set up.
Getting going with log4net
All the previous posts were about things I've done (and not everything I've done - they'll make nice future filler). I'll be typing this one up as I go through the learning curve, so there should be more detail for you bit-heads out there.
I've used log4net, and variations thereof (there's a log4*, it seems). Logging facilities and levels are very configurable, and adaptable to almost any need.
For example (and you'll see this one everywhere):
An "appender" is your output, the console in this case. You can get color console output, if you like (love it!). There's also files, rolling files, Event Log, database, SMTP, and more appenders.
Next is our "layout" - what do we want the log entries to look like? This uses the very flexible "PatternLayout", which lets you use "%whatever" to format the output.
Then you've got your "root" element. These are the default logger settings.
Next, we have named logger, "Spring". An application can use this log by name, and its output can be somewhere else entirely.
From the developer's standpoint, things look like this:
They don't have to know where they're writing their messages, or whether or not they are writing their message at all. If everybody uses named loggers, then you, the poor IT guy who gets called when it all breaks down, don't have to waste your time waiting for a developer to wake up, too. Just crank up logging wherever it seems appropriate, and dig on in.
Even better, when it comes time to move it, the necessary changes are obvious.
I know that there a lot of shops out there that will claim to be doing this. Many of them are, if "including the library in the project" counts. If you're not seeing this level of granularity, question why, and insist on it. It doesn't have to be log4net, but it should be very configurable.
Back on topic...
O'Reilly provides a nice tutorial.
The real dirt can be found at log4net's home at the ASF.
I've used log4net, and variations thereof (there's a log4*, it seems). Logging facilities and levels are very configurable, and adaptable to almost any need.
For example (and you'll see this one everywhere):
<log4net>
<appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="ConsoleAppender" />
</root>
<logger name="Spring">
<level value="INFO" />
<appender-ref ref="ConsoleAppender" />
</logger>
</log4net>
An "appender" is your output, the console in this case. You can get color console output, if you like (love it!). There's also files, rolling files, Event Log, database, SMTP, and more appenders.
Next is our "layout" - what do we want the log entries to look like? This uses the very flexible "PatternLayout", which lets you use "%whatever" to format the output.
Then you've got your "root" element. These are the default logger settings.
Next, we have named logger, "Spring". An application can use this log by name, and its output can be somewhere else entirely.
From the developer's standpoint, things look like this:
Logger.Info( "informative message");
Logger.Warn( "warning message" );
Logger.Error( "error message" );
They don't have to know where they're writing their messages, or whether or not they are writing their message at all. If everybody uses named loggers, then you, the poor IT guy who gets called when it all breaks down, don't have to waste your time waiting for a developer to wake up, too. Just crank up logging wherever it seems appropriate, and dig on in.
Even better, when it comes time to move it, the necessary changes are obvious.
I know that there a lot of shops out there that will claim to be doing this. Many of them are, if "including the library in the project" counts. If you're not seeing this level of granularity, question why, and insist on it. It doesn't have to be log4net, but it should be very configurable.
Back on topic...
O'Reilly provides a nice tutorial.
The real dirt can be found at log4net's home at the ASF.
Thursday, April 16, 2009
Solr Indexing
Rather than go the DBMS route for my file index, I decided to go with Solr, from the Apache Foundation. I want to keep the index away from the data - for what I have in mind, they are separate things. It doesn't mean there won't be a regular database sitting around someplace, with its own indexes, it's just that I wanted this specific index to stand on its own.
Solr does everything that I would've written, anyway. It accepts and returns data over HTTP, as XML or JSON. Replication, caching, and a bunch of other shiny and fun things. Since it is running as a Java servlet, there's all kinds of stuff you stick in its way, and mega-configurable.
Retrieving data is pretty fast. Insertions, not so much. There isn't much data going back and forth, and I realize that insertions are, by their nature, slow. I suspect I could get better performance with fewer indexed elements, spread out among solr instances (the Servlet aspect makes this pretty easy).
One of the slowdowns I've pinpointed has to do with the .NET WebClient class. It sucks. It uses some Windows-provided HTTP API, and it isn't happy with multiple threads. It also limits itself to two connections at once (then appears to deadlock - but it might be my code). The HTTP specification says only two connections at once, and I think there's a way to override this in the Windows' registry.
Not interested in that path, especially if it is going to be this slow.
At the moment, there's a commit that occurs after every add. This is another slowdown, but the library I'm using, SolrSharp, doesn't make doing it another way very pretty. It shouldn't be too hard to fix - I've been dinking around in the code, and I see what changes I have to make. Its just a lot of refactoring. Bleh.
So, on the todo list are:
Solr does everything that I would've written, anyway. It accepts and returns data over HTTP, as XML or JSON. Replication, caching, and a bunch of other shiny and fun things. Since it is running as a Java servlet, there's all kinds of stuff you stick in its way, and mega-configurable.
Retrieving data is pretty fast. Insertions, not so much. There isn't much data going back and forth, and I realize that insertions are, by their nature, slow. I suspect I could get better performance with fewer indexed elements, spread out among solr instances (the Servlet aspect makes this pretty easy).
One of the slowdowns I've pinpointed has to do with the .NET WebClient class. It sucks. It uses some Windows-provided HTTP API, and it isn't happy with multiple threads. It also limits itself to two connections at once (then appears to deadlock - but it might be my code). The HTTP specification says only two connections at once, and I think there's a way to override this in the Windows' registry.
Not interested in that path, especially if it is going to be this slow.
At the moment, there's a commit that occurs after every add. This is another slowdown, but the library I'm using, SolrSharp, doesn't make doing it another way very pretty. It shouldn't be too hard to fix - I've been dinking around in the code, and I see what changes I have to make. Its just a lot of refactoring. Bleh.
So, on the todo list are:
- a better HTTP client
- modify the addition of records to allow batching
Spring.NET - It's Shiny! I Love It!
The boring bit about what Spring.NET does, from their overview page:
Truth is, there's a lot to the whole Spring.NET project, a lot more than I'm going to cover here. Basically, its a code framework, or a set of utilities to simplify writing code. Why would a self-professed IT goon be interested in a code framework? Let me show you how I've been using it.
The main point of interest, for now, is the IoC container. I'd like to copy and paste what that's about, but the Spring.NET punts on that, and sends you to a couple of other pages. The short version is that rather than having one class "know" a lot about another class - specifically initialization, in my case - there exists an outside "container" which knows about all the parts, and pulls them together. In a best case scenario, the classes themselves know little about their dependencies.
Here's a pseudo-config:
Given that, when the code calls (abbreviated):
It will not only get a "CustomClass.Thing", it will get it with the two properties set to the specified values, even private properties. There's variations for all kinds of objects and values. You can even use another object specified in the same config.
Let's put our IT hats on for a minute, and see what this gives us: configurability, without having to go ask some developer to make a change. All they have to do is expose anything remotely configurable as a property, and you only need to bother them for the inevitable bugs. If you don't think this is a big deal, then you haven't ever tried to get an enterprise development group to make a config change.
As a developer, you can spend a lot less time worrying about how your objects are going to get their configuration details. The calling object doesn't worry about it, the receiving object doesn't worry about it. It's like magic, only it's not.
Spring.NET provides comprehensive infrastructural support for developing enterprise .NET applications. It allows you to remove incidental complexity when using the base class libraries makes best practices, such as test driven development, easy practices. Spring.NET is created, supported and sustained by SpringSource.This is how you know that Spring.NET is ready for the enterprise: a whole paragraph which doesn't actually tell you anything. (and for you java weenies out there that are getting ready to go on and on about how its from java: I know)
Truth is, there's a lot to the whole Spring.NET project, a lot more than I'm going to cover here. Basically, its a code framework, or a set of utilities to simplify writing code. Why would a self-professed IT goon be interested in a code framework? Let me show you how I've been using it.
The main point of interest, for now, is the IoC container. I'd like to copy and paste what that's about, but the Spring.NET punts on that, and sends you to a couple of other pages. The short version is that rather than having one class "know" a lot about another class - specifically initialization, in my case - there exists an outside "container" which knows about all the parts, and pulls them together. In a best case scenario, the classes themselves know little about their dependencies.
Here's a pseudo-config:
<spring>
<objects>
<object name="CustomClassThing" type="CustomClass.Thing, CustomClass">
<property name="SomePublicProperty" value="meh">
<property name="SomePrivateProperty" value="heh">
</object>
</objects>
</spring>
Given that, when the code calls (abbreviated):
Thing thing = context.GetObject("CustomClassThing");
It will not only get a "CustomClass.Thing", it will get it with the two properties set to the specified values, even private properties. There's variations for all kinds of objects and values. You can even use another object specified in the same config.
Let's put our IT hats on for a minute, and see what this gives us: configurability, without having to go ask some developer to make a change. All they have to do is expose anything remotely configurable as a property, and you only need to bother them for the inevitable bugs. If you don't think this is a big deal, then you haven't ever tried to get an enterprise development group to make a config change.
As a developer, you can spend a lot less time worrying about how your objects are going to get their configuration details. The calling object doesn't worry about it, the receiving object doesn't worry about it. It's like magic, only it's not.
The Project
The project of the moment is yet another file indexer. This one, of course, will revolutionize everything and make me rich because there is no way anybody else has ever thought to do this. That's irrelevant, though.
What is relevant is what I'm learning as I go. I wanted to learn about the latest C# features - the lambda syntax (rocks!), LINQ (sucks!), WCF (what is this supposed to be improving?), ASP.NET MVC (about f'n time!), and just generally come to grips with Microsoft's ecosystem's latest and greatest.
This includes open source stuff. So far, my experience has been that MS is pretty good at their core tech, but a lot of their other tech is second-rate to the OSS alternatives. Indeed, they are usually playing "follow the leader" - ASP.NET MVC for example. Nor am I wedded to the MS platform, in general; the actual index is Solr running in a zone on OpenSolaris.
Where I've been so far:
Why not Ruby, Python, or any of their associated frameworks? Well, I've done a bit of Rails work, and a lot of scripting in Ruby. I've done some in Python. They're great, and I will use them again in the future, I'm sure. Right now, I'm behind on MS technology. I don't think I do myself any favors by ignoring a significant chunk of the technology world.
The verdict so far? Not bad, not bad at all. I've let myself go through the "It's shiny! I love it! But it smells horrible! I hate it! With a little perfume, it's tolerable" cycle with a few things, and I'm okay with what I'm seeing.
Powershell, for example, is a genuine game-changer for the Windows platform - that hasn't kept it from pissing me off more than once. CodePlex sucked from the word "go" (I won't even bother linking to it - I'd rather spend the time writing a parenthetical expression of how I'm not going to link to it, it sucked so hard).
If you dive head-first into the MS Kool-Aid, you are taken care of quite well. The integration of everything is really nice. Yes, I've used Eclipse and NetBeans. No, they aren't nearly as good as VS2008. Seriously. It isn't until you step out of whatever usage scenarios that MS anticipated that you have any problems. (CodePlex is an example - I don't want to use TFS, it brings "suck" to "vacuum of intergalactic space" proportions.).
That's where things stand. I'll be updating in a bit, and we'll talk about Spring.NET.
What is relevant is what I'm learning as I go. I wanted to learn about the latest C# features - the lambda syntax (rocks!), LINQ (sucks!), WCF (what is this supposed to be improving?), ASP.NET MVC (about f'n time!), and just generally come to grips with Microsoft's ecosystem's latest and greatest.
This includes open source stuff. So far, my experience has been that MS is pretty good at their core tech, but a lot of their other tech is second-rate to the OSS alternatives. Indeed, they are usually playing "follow the leader" - ASP.NET MVC for example. Nor am I wedded to the MS platform, in general; the actual index is Solr running in a zone on OpenSolaris.
Where I've been so far:
- PowerShell
- Windows 2008
- Visual Studio 2008
- Vista
- CodePlex
- git
- Solr
- Solaris
Why not Ruby, Python, or any of their associated frameworks? Well, I've done a bit of Rails work, and a lot of scripting in Ruby. I've done some in Python. They're great, and I will use them again in the future, I'm sure. Right now, I'm behind on MS technology. I don't think I do myself any favors by ignoring a significant chunk of the technology world.
The verdict so far? Not bad, not bad at all. I've let myself go through the "It's shiny! I love it! But it smells horrible! I hate it! With a little perfume, it's tolerable" cycle with a few things, and I'm okay with what I'm seeing.
Powershell, for example, is a genuine game-changer for the Windows platform - that hasn't kept it from pissing me off more than once. CodePlex sucked from the word "go" (I won't even bother linking to it - I'd rather spend the time writing a parenthetical expression of how I'm not going to link to it, it sucked so hard).
If you dive head-first into the MS Kool-Aid, you are taken care of quite well. The integration of everything is really nice. Yes, I've used Eclipse and NetBeans. No, they aren't nearly as good as VS2008. Seriously. It isn't until you step out of whatever usage scenarios that MS anticipated that you have any problems. (CodePlex is an example - I don't want to use TFS, it brings "suck" to "vacuum of intergalactic space" proportions.).
That's where things stand. I'll be updating in a bit, and we'll talk about Spring.NET.
Intro
Ho hum, another day, another new blog on the internet.
This particular one is about my own experiences in the IT world, and related ruminations. Basically, I was told I need to blog this stuff, so here I am.
I don't have a lot of fancy letters after my name, but I do have a lot of time sitting behind a keyboard, screwing around with other peoples' computers. A lot of time.
From the looks of things, I'm going to be doing a lot of writing about Microsoft products. Right about now, I'm either supposed to apologize, and explain that I'd much rather be doing things on a Superdome running HPUX, or go all fanboi about Windows! Windows! Windows!
Meh.
The way things are right now, I'm writing code in C#. It's what my little market wants, so that's what they get.
What do I do? Right now, a whole lotta nuthin'. Good thing I'm good at it. I've got a little side project I've been working on, and that's what I'll be babbling about for now...
This particular one is about my own experiences in the IT world, and related ruminations. Basically, I was told I need to blog this stuff, so here I am.
I don't have a lot of fancy letters after my name, but I do have a lot of time sitting behind a keyboard, screwing around with other peoples' computers. A lot of time.
From the looks of things, I'm going to be doing a lot of writing about Microsoft products. Right about now, I'm either supposed to apologize, and explain that I'd much rather be doing things on a Superdome running HPUX, or go all fanboi about Windows! Windows! Windows!
Meh.
The way things are right now, I'm writing code in C#. It's what my little market wants, so that's what they get.
What do I do? Right now, a whole lotta nuthin'. Good thing I'm good at it. I've got a little side project I've been working on, and that's what I'll be babbling about for now...
Subscribe to:
Posts (Atom)