Why Programmer Must Have Math Background

Now I definitely sure that each programmer (or software developer, or software engineer, call he/she as you want) must have math background.

Lack of analytical skills will lead to unclear, tangled, repetitious, inconsistent code. Let’s consider simple example – from web development area (which is believed to be not so complicated, although this is, of course, arguable) – we have several custom controls which are based on same HTML elements which are, in their turn, transformed into aforementioned controls by applying to them some jQuery plugin. Person without, so to say, “pattern recognition” skill will code something like this on different pages: $(“#cars”).slider(), $(“#bikes”).slider() and so on, instead of just writing $(“.slider”).slider(). More depressing case is when it comes to design – you’ll see lots of similar code snippets (with only slight differences which can be eliminated via parameterizing). Of course, even experienced developers can end up with unclear architecture, but this is definitely easier when they do not have analytical skills which are developed, I strongly believe, while doing math excercises.

Maybe you’ve already considered me to be Captain Obvious, but, since I still can hear from here and there: “Why do I need to do these stupid exercises in algebra/trigonometry/differential calculus/your option instead of just learning jQuery?” I think this post to some extent can be useful.

ASP.NET MVC – Preparing for Deployment

Development process of several of my pet projects reached stage when robust deployment process is needed. So, I decided to consider list of options and started googling.

I found that msbuild itself allows to do packaging for deployment (but very few developers were recommending to use it for real-life deployment).

Other option was use of NAnt – but disadvantages were that it needs to write configuration in XML, which is quite verbose way of expressing things, and also it has little or no development activity as for now.

But then I spotted psake which seemed to be like rake, but for PowerShell, not Ruby. I even read nice post with real life example from Ayende.

So, decision was made – packaging will be done by script written in PowerShell. Advantages:

• no need in additional interpreter/dependencies (as in case with rake)
• ability to use .NET types from script
• quite expressive language and decent syntax
• no need to write tons of XML

So, for one of projects, I even didn’t use psake and expressed everything in plain PowerShell with only dependency on 7-zip for creating zip archive and it’s only few lines of clean and understandable code – you can look at it here, on Github (it’s well commented, so it’s good place to start if you haven’t written such things before).

NullReferenceException from the Entity Framework Core

Running web application with Entity Framework 4.3 I got very strange (at first glance) exception saying: “Object reference not set to an instance of an object.”. Following is the controller action caused this error:

public ActionResult Index()
{
    return View(repository.Query<Story>().ToList());
    // repository.Query<Story>() here is nothing but (DbQuery<Story>) context.Stories
}

Here is the stack trace:

[NullReferenceException: Object reference not set to an instance of an object.]
   System.Data.Objects.ELinq.QueryParameterExpression.EvaluateParameter(Object[] arguments) +83
   System.Data.Objects.ELinq.ELinqQueryState.GetExecutionPlan(Nullable`1 forMergeOption) +779
   System.Data.Objects.ObjectQuery`1.GetResults(Nullable`1 forMergeOption) +149
   System.Data.Objects.ObjectQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator() +44
   System.Data.Entity.Internal.Linq.InternalQuery`1.GetEnumerator() +40
   System.Data.Entity.Infrastructure.DbQuery`1.System.Collections.Generic.IEnumerable<TResult>.GetEnumerator() +40
   System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) +315
   System.Linq.Enumerable.ToList(IEnumerable`1 source) +58
   QTeam.PlanningBoard.Controllers.StoriesController.Index() in ...QTeam.PlanningBoard\Controllers\StoriesController.cs:26
   lambda_method(Closure , ControllerBase , Object[] ) +97
   System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17
   System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters) +208
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +27
   System.Web.Mvc.<>c__DisplayClass15.<InvokeActionMethodWithFilters>b__12() +55
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation) +263
   System.Web.Mvc.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14() +19
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +191
   System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +343
   System.Web.Mvc.Controller.ExecuteCore() +116
   System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +97
   System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext) +10
   System.Web.Mvc.<>c__DisplayClassb.<BeginProcessRequest>b__5() +37
   System.Web.Mvc.Async.<>c__DisplayClass1.<MakeVoidDelegate>b__0() +21
   System.Web.Mvc.Async.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _) +12
   System.Web.Mvc.Async.WrappedAsyncResult`1.End() +62
   System.Web.Mvc.<>c__DisplayClasse.<EndProcessRequest>b__d() +50
   System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f) +7
   System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action) +22
   System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +60
   System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) +9
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +8970061
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +184

Reading such stack trace can mislead you that cause is not in app code but somewhere in the EF itself. This is because lazy enumeration – after a little bit deeper analysis I found that I have such code:

/// <summary>
/// Queries data for current tenant.
/// </summary>
private IQueryable<T> QueryFilteredInternal<T>(params Expression<Func<T, object>>[] eagerlyLoadedProperties) where T : class, IIdentifiable, IOwned
{
    return repository
        .Query(eagerlyLoadedProperties)
        .Where(x => x.OwnerId == userService.CurrentUser.Id);
}

Here userService.CurrentUser was null, so when EF tried to access userService.CurrentUser.Id for providing parameter value to query, NullReferenceException was occuring.

So, summary is: don’t fully rely on call stack investigating cause for exceptions while working with lazy evaluated things.

P. S. In the early beginning first thing I did was googling for stack trace. It gave me nothing but links to failing sites and not to articles with solutions, as was expected, so I hope this post will be helpful for someone who will issue similar search query.

ASP.NET File Upload Possible Issue

One of issues you can face when developing web application with use of ASP.NET MVC and IIS 7.5 is maximal request size. No doubt, after googling, you’ll find that cause is request size limit, so you’ll happily add to web.config following line and forget about this.

<httpRuntime maxRequestLength="100000"/>

But if under some circumstances you’ll find that controller’s action responsible for receiving of large file is not being triggered but instead application returns 404 code, try following solution. Please note, in this case content length should be provided in bytes, not kilobytes like in prevous example.

<security>
    <requestFiltering>
        <requestLimits maxAllowedContentLength="100000000" />
    </requestFiltering>
</security>

For, me, while working with Windows integrated authentication, this helped a lot. Thanks to StackOverflow!

Snapping To Grid

Let’s suppose user selects some value (real number) and you need to snap it to nearest point on discrete axis. No problem!

public double SnapValue(double value, double step)
{
    var remainder = Math.IEEERemainder(value, step);
    return step/2 > remainder ? value - remainder : value + remainder;
}

So, assuming your you have an axis with step 0.5, value 10.2 will be snapped to 10 (closest node), and value 10.78, in its turn, – to 11.

When IList Has Indexer But Does Not Have Enumerator

What can you expect from type implementing IList? Surely, availability of enumerator, which give you a power of LINQ! But do not be so fast, sometimes this assumption can be wrong! For example, yesterday I’ve found that some types in XAF (eXpress Application Framework by DevExpress) can implement IList having the indexer but at the same time no enumerator!

Imagine you are implementing handler for some controller’s action where you need to access all items from ListView:

...
var records = listView.CollectionSource.List.Cast<MyType>().ToList();
...

Instance listView.CollectionSource.List is of type implementing IList, so you can expect reference to list of MyType instances in the records variable, but you’ll receive System.NotSupportedException!

  System.NotSupportedException was unhandled by user code
  Message=Specified method is not supported.
  Source=DevExpress.Xpo.v10.2
  StackTrace:
       at DevExpress.Xpo.Helpers.XpoServerCollectionAdderRemover.GetEnumerator()
       at DevExpress.Xpo.Helpers.XpoServerCollectionWrapperBase.GetEnumerator()
       at System.Linq.Enumerable.<CastIterator>d__aa`1.MoveNext()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at MyProject.MyController.HandleMyAction(Object sender, SimpleActionExecuteEventArgs e)
       at DevExpress.ExpressApp.Actions.SimpleAction.RaiseExecute(ActionBaseEventArgs eventArgs)
       at DevExpress.ExpressApp.Actions.ActionBase.ExecuteCore(Delegate handler, ActionBaseEventArgs eventArgs)

So, what should you do? Use indexer!

...
var records = Enumerable.Range(0, listView.CollectionSource.List.Count).Select(
  index => listView.CollectionSource.List[index] as MyType).ToList();
...

Is it so hard for framework developers to implement enumerator if type already has indexer? I think no. So it’s very strange that we should bother ourselves with such things. But anyway the solution is not so hard, so generally everything is ok 🙂 .

P. S. Also it’s quite strange and unintuitive that we should access items of list view through listView.CollectionSource.List instead of listView.List or listView.Items, especially, when such properties do exist, but intended for another purposes (internal). But this is another story.

XPath: Select Node with Child Containing Given Substring

XPath is very interesting topic, actually it allows you to do quite sophisticated things. And in this short post, I’ll give example how to create a little more complex than plain query: it will return all books, which contains “C++” in any of its childrens’ texts (please, see example of such XML here).

So, our query is next:

//*[contains(text(), 'C++')]/parent::book

It will return:

<book id="bk112">
    <author>Galos, Mike</author>
    <title>Visual Studio 7: A Comprehensive Guide</title>
    <genre>Computer</genre>
    <price>49.95</price>
    <publish_date>2001-04-16</publish_date>
    <description>Microsoft Visual Studio 7 is explored in depth,
    looking at how Visual Basic, Visual C++, C#, and ASP+ are 
    integrated into a comprehensive development 
    environment.</description>
</book>

That’s it. Simple. But useful, especially if you haven’t worked with XPath too much before.

Dealing with Namespaces in XPath (.NET)

This post is kind of short but useful note for those who are dealing with namespaces in XPath, especially if there is default namespace (with empty prefix).

Normally, when doing queries to document with namespaces, you involve XmlNamespaceManager like this:

var namespaceManager = new XmlNamespaceManager(document.NameTable);

var namespaces = document
    .DocumentElement
    .Attributes
    .Cast<XmlAttribute>()
    .Where(x => x.Name.StartsWith("xmlns"))
    .Select(x => new
        {
            Prefix = x.Name.Length > 5
                            ? x.Name.Substring(6) // case when name is like "xmlns:something"
                            : string.Empty, // case when name is "xmlns" - default namespace
            Uri = x.Value
        });

foreach (var ns in namespaces)
{
    namespaceManager.AddNamespace(ns.Prefix, ns.Uri);
}

var nodes = document.SelectNodes("//node/subnode", namespaceManager);

But wait! This query will return no results! The reason is not so obvious: you must use prefix for default namespace, it can’t be omitted! So, to make such queries working, we need to change the initial code as well as XPath query:

...
            Prefix = x.Name.Length > 5
                            ? x.Name.Substring(6) // case when name is like "xmlns:something"
                            : "default", // case when name is "xmlns" - default namespace
            Uri = x.Value
...
var nodes = document.SelectNodes("//default:node/default:subnode", namespaceManager);

Good, now it’s ok, we’ve got our results.

Extracting Tables from Microsoft Word 2007+ (OpenXML) Document

Let’s suppose we have huge Word 2007+ document (*.docx) with lots of tables and we want to extract that information for further processing. Since our document is based on OpenXML format I would say “No problem! Let’s do it!” :).

First, we need to open zip archive (each *.docx is zipped set of XML files). For this purpose we can use great library DotNetZip (just use NuGet to obtain it). Right after this we’ll read word/document.xml entry.

static void Main(string[] args)
{
    using (var file = ZipFile.Read(args[0]))
    using (var inputStream = file["word/document.xml"].OpenReader())
    {
        var document = new XmlDocument();
        document.Load(inputStream);

        var tables = ExtractTables(document);

        using (var outputStream = File.Create(args[1]))
        {
            new XmlSerializer(typeof(Table[])).Serialize(outputStream, tables);
        }
    }
}

Then we’ll extract neccessary pieces using XPath: our target is w:tbl nodes (tables) with such descendants as w:tr (rows), w:tc (cells) and w:t (text pieces).

private static Table[] ExtractTables(XmlNode document)
{
    return document.Select("//w:tbl", tableNode => new Table
    {
        Rows = tableNode.Select(".//w:tr", rowNode => new Row
        {
            Cells = rowNode.Select(".//w:tc", cellNode => new Cell
            {
                Text = cellNode.Select(".//w:t", textNode => textNode.InnerText)
            })
        })
    });
}

Classes Table, Row and Cell are simple DTOs used for simplifying of result serialization.

One more thing is .Select() extension method – it’s implemented in the next way:

public static T[] Select<T>(this XmlNode node, string xpath, Func<XmlNode, T> selector)
{
    return node
        .SelectNodes(xpath, GetOrCreateNamespaceManager((node as XmlDocument ?? node.OwnerDocument).NameTable))
        .Cast<XmlNode>()
        .Select(selector)
        .ToArray();
}

Method .GetOrCreateNamespaceManager() returns namespace manager as it’s required for making of XPath queries with use of namespaces. There is only one parameter for it – name table instance.

private static XmlNamespaceManager GetOrCreateNamespaceManager(XmlNameTable nameTable)
{
    if (manager == null)
    {
        manager = new XmlNamespaceManager(nameTable);
        manager.AddNamespace("w", "http://schemas.openxmlformats.org/wordprocessingml/2006/main");
    }

    return manager;
}

That’s all! Due to openness of new Word document’s format our initial task is so simple in implementation that it takes no more than couple of hours to implement it. Nice! 🙂

BDD with SpecFlow and WatiN

I’ve long heard about BDD, and recently viewed lectures from SaaS class really interested me not only in Ruby (with such great frameworks as Rails and Cucumber), but also in practical use of this methodology in .NET, which is my main platform at the moment. So, I decided to try writing of small web app using BDD approach assisted by SpecFlow tool for definition of features and latter execution of test based on them, and WatiN as tool which would help to grab content from browser to make assertions on expected results.

Here is the feature:

Feature: Viewing of rates
	In order to view available conversion rates
	As a user
	I want to see current rates as well as their changes history

Scenario: View current rates
	Given I have following exchange rates in the system
		| From | To  | Rate |
		| USD  | UAH | 8.03 |
		| RUR  | UAH | 0.27 |
	When I am on home page
	Then I should see following rows
		| V1  | V2  | V3     |
		| USD | UAH | 8.0300 |
		| RUR | UAH | 0.2700 |

Implementation of most steps is pretty straightforward and not listed here, except “Then” step. So, below is initial implementation of this step.

[Then(@"I should see following rows")]
public void ThenIShouldSeeFollowingRows(Table table)
{
    foreach (var row in table.Rows)
    {
        Assert.That(
            Browser.Table("exchange-rates").TableRows.Any(
                exchangeRate => row.Values.All(
                    text => exchangeRate.TableCell(Find.ByText(text)).Exists)),
            string.Format("Page contains following row: {0}.", row.ToReadableString()));
    }
}

Being WatiN newbie, I’d made a step on a rake and trial to execute scenario led to StackOverflowException in WatiN.Core, so after googling a bit I decided to parse table first and only then make assertions. Said – done: updated implementation is below.

[Then(@"I should see following rows")]
public void ThenIShouldSeeFollowingRows(Table table)
{
    var actualTable = Browser
        .Table("exchange-rates")
        .TableRows
        .Where(r => r.TableCells.Any())
        .Select(r => r.TableCells.Select(c => c.Text.Trim()).ToArray())
        .ToArray();

    foreach (var row in table.Rows)
    {
        Assert.That(
            actualTable.Any(r => row.All(v => r.Contains(v.Value))),
            string.Format("Page contains following row: {0}.", row.ToReadableString()));
    }
}

Please note that Trim() call on text of cell: it’s important since text can be, for example, with trailing space (as it was in my case).

So, finally I can get to actual implementation of this feature and then, running the above scenario, obtain long-awaited result.

ViewCurrentRates : Passed
Given I have following exchange rates in the system
  --- table step argument ---
  | From | To  | Rate |
  | USD  | UAH | 8.03 |
  | RUR  | UAH | 0.27 |
-> done: DataPreparationSteps.GivenIHaveFollowingExchangeRatesInTheSystem(<table>) (7,9s)
When I am on home page
-> done: NavigationSteps.WhenIAmOnPage("home") (5,1s)
Then I should see following rows
  --- table step argument ---
  | V1  | V2  | V3     |
  | USD | UAH | 8.0300 |
  | RUR | UAH | 0.2700 |
-> done: VerificationSteps.ThenIShouldSeeFollowingRows(<table>) (0,4s)

Overall impressions: even if it adds some difficulties (especially at the beginning), I think it’s great idea to practice BDD, and as for tooling: both SpecFlow and WatiN are nice and intuitive, and it cost quite low effort to get familiar with them (at least in this certain case).