Tuesday, January 1, 2013

Oswos.net

Some of you may already have seen Oswos, my (not all too bad) attempt to create a simple webserver for a netbook. Now the performance is nothing compared to Nginx on a Debian server. But I've learned a lot during the development of this.


One thing that came to my mind while working on Oswos was the interest for server management. I wanted Oswos to contain an easy to maintain server management tool. But was far to busy getting bugs out of the webserver. That's why I descided to build some simple open source tools and applications to help you maintain your server and keep your sites up and running.

The first tool I'm building is an IP Monitoring tool. Why? Because I host a couple of websites on my dynamic IP webserver and want to be notified when the IP address changes so I can update the DNS.

This project contains 2 parts:
1. A simple python script which you can install on your server. This script poll's every x minutes if the IP has changed and pushes a message to your phone.
2. A simple Android application which receives the push notifications from the python script and displays the notification.

The android application is almost ready for deployment on GitHub, the Python scripts need an implementation of the push service for Android. Once the prototype is working, I'll give an update on this blog. After that I'll be focussing on the simple deployment and installation of this (and other) scripts.
I'm really curious about what you think of this idea, maybe have some input, or want to help set this "brand for server tools" up.

Also check out the new layout of www.luuk.it and tell me if you like it.

Best wishes and have a creative, meaningfull 2013 by doing the stuff you love.

Luuk

Wednesday, November 21, 2012

Tfs Work Item HtmlFieldControl height on sharepoint

During my investigation to create a customer friendly interface for Tfs Work Items, we've found an interesting 'hidden' feature: the Height attribute. Without it, the HtmlFieldControl take up a huuuuuuuuuge amount of space:


But by adding the super secret nowhere to be found Height attribute, magic happens...


*badadum*

<Tab Label="Issue Details">
  <Group>
    <Column PercentWidth="100">
      <Control FieldName="WhatWentWrong" Type="HtmlFieldControl" Label="What went wrong:" LabelPosition="Top" Height="75" />
    </Column>
  </Group>
  <Group>
    <Column PercentWidth="100">
      <Control FieldName="ReproduceSteps" Type="HtmlFieldControl" Label="Steps to reproduce the problem:" LabelPosition="Top" Height="75" />
    </Column>
  </Group>
  <Group>
    <Column PercentWidth="100">
      <Control FieldName="ExpectedBehavior" Type="HtmlFieldControl" Label="What is the expected behavior:" LabelPosition="Top" Height="75" />
    </Column>
  </Group>
</Tab>
By the way, to prevent the Html controls, make te type of the field PlainText.

Cheers!

Thursday, November 8, 2012

SSDT Script not found and update button disabled

SSTD is an awesome tool for comparing databases and updating them. Allthough it has some performance problems with the Object Definitions window open, it is still cool.

Yesterday I had some annoying problems with it. The first one I encountered was a message called 'Script not found':





It was caused by the .refactorlog in the project. After deleting the file the errormessage was gone.

But now what... the update and script buttons are disabled... lets click compare again and again. Nothing enables it. After some testing it was caused by a compile error which was solved with the compare window open. Lesson learned: After solving the compile error, you'll have to close the schema compare window and open a fresh one. Now they are enabled again.

Saturday, October 6, 2012

Export and import a large set of resx resources (part 2)

Finally I had some time to build the import of the ResourceExtractor code. Allthough this wasn't that hard to complete, I've been very busy with moving to my new house and getting settled.

Now you can simply extract resx files to xls, and merge them back to resx (you can even use this with the prebuild command). All the usage options can be found in the help (by running the executable without any arguments).

Off cource, all of this goodness can be found at https://github.com/luuksommers/resourceextractor Beware it is build in VS2012! And once you'll get used to the color scheme you'll love it!

If you have questions or comments please leave them below!
Cheers,
L

Removing enters from excel

Just to remind myself when someone delivers crappy import xls files:

1) Press CTRL+H (i.e. shortcut for Find and Replace)
2) Click in the ‘Find what‘ field, hold down the ALT key and type 0010;. It may not seem like anything happened but you actually entered an invisible line break character
3) At this point you can choose to Replace All.

Tuesday, April 17, 2012

SQL Audit Trail with pass through user credentials

For a customer we needed to add an audit trail to the database.

When you first start looking on the internet, you will probably find this link: http://www.nigelrivett.net/AuditTrailTrigger.html. The mayor problem with this audit trail is that the table will be huge because it stores the change of every field in a table in a seperate row. For our audit trail we needed a more simple approach: Just a shadow copy of the record which was edited. After some investigation I've found the build in SQL feature called 'Change Data Capture', which is only available in the enterprise edition. This is a little above our budget of 0 euro (as usual in IT it may cost nothing) ;). The other problem I found with Change Data Capture is that it isn't transparent enough. It does it's job, but nobody knows exactly how it works, I find this a bit tricky for our production environment.

So after just some more (2 hours!) searching I've come to the magnificent piece of SQL you'll find below from http://www.codeproject.com/Articles/21068/Audit-Trail-Generator-for-Microsoft-SQL.

After some minor adjustments it was ready to go!

But wait...

Our website users don't connect directly to the database, they use LINQ to SQL which has the user credentials of our Application Pool, so we still can't see who did the change :( StackOverflow has the answer: http://stackoverflow.com/questions/3197607/how-to-pass-out-of-band-current-user-id-data-to-sql-server-2008. Apparently there is an option to set CONTEXT_INFO on a connection so we can pass some extra data from the website user.

Time to hack LINQ! We want to execute the following statement everytime a new connection is used by the user:
SET CONTEXT_INFO 0x<binaryusername>

First we create a base class for our DataContext on which we fetch all Connection StateChange events. When the connection is opened send the 'SET CONTEXT_INFO' statement.
public class DataContextAuditBase
{
    public DataContextAuditBase(string connection) : base(connection)
    {
        Connection.StateChange += ConnectionStateChange;
    }

    public DataContextAuditBase(IDbConnection connection) : base(connection)
    {
        Connection.StateChange += ConnectionStateChange;
    }

    public DataContextAuditBase(string connection,
             MappingSource mappingSource) : base(connection, mappingSource)
    {
        Connection.StateChange += ConnectionStateChange;
    }

    public DataContextAuditBase(IDbConnection connection,
             MappingSource mappingSource) : base(connection, mappingSource)
    {
        Connection.StateChange += ConnectionStateChange;
    }

    private string UserName
    {
        get { return (HttpContext.Current != null) ? HttpContext.Current.User.Identity.Name : Environment.UserName; }
    }

    private string StringToHex(string convert)
    {
        System.Text.ASCIIEncoding  encoding = new System.Text.ASCIIEncoding();
        return BitConverter.ToString(encoding.GetBytes(convert)).Replace("-","").ToLower();
    }

    private void ConnectionStateChange(object sender, StateChangeEventArgs e)
    {
        if( e.CurrentState == ConnectionState.Open)
        {
            ExecuteCommand("SET CONTEXT_INFO 0x" + StringToHex(UserName));
        }
    }
}

And change the BaseClass of the DataContext:


Run the newly created website and see the magic happening, right before your eyes!


The used audit trail stored procedure (used to create tables on which you want to use the audit):
/******************************
** Name: sp_GenerateAuditTable
** Desc: creates an audit table in a database of  your choosing from a user 
**       table that you specify in whatever database you run the sproc against
** Original Author: Cedric Baelemans
** Modifying Author: Andrew Tappert (2011-3-17)
** Modifying Author: Luuk Sommers (2012-04-17)
** Date: 2012-04-17
** 
** Found on: http://sameproblemmorecode.blogspot.com/2012/04/sql-audit-trail-with-pass-through-user.html
** More info on original: http://www.codeproject.com/Articles/21068/Audit-Trail-Generator-for-Microsoft-SQL
**************************/
ALTER PROCEDURE [dbo].[sp_GenerateAuditTable]
 @TableName varchar(128),
 @Owner varchar(128) = 'dbo',
 @AuditNameExtension varchar(128) = '_shadow',
 @DropAuditTable bit = 0,
 @AuditDatabaseName varchar(128) = null
AS
BEGIN
 declare @sql nvarchar(4000)
 
 if not exists( SELECT schema_name FROM information_schema.schemata WHERE schema_name = @Owner )
 BEGIN
  PRINT 'Creating chema [' + @Owner + ']'
  set @sql = 'CREATE SCHEMA [' + @Owner + '] AUTHORIZATION [dbo]'
  print @sql
  EXEC (@sql)
 END

 -- Check if table exists
 IF not exists (SELECT * FROM dbo.sysobjects WHERE id = object_id(N'[' + @Owner + '].[' + @TableName + ']') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
 BEGIN
  PRINT 'ERROR: Table does not exist'
  RETURN
 END
 
 IF @AuditDatabaseName is null
 BEGIN
   set @AuditDatabaseName = (select DB_NAME())
 END
 
 -- Check @AuditNameExtension
 IF @AuditNameExtension is null
 BEGIN
  PRINT 'ERROR: @AuditNameExtension cannot be null'
  RETURN
 END
 
 -- Drop audit table if it exists and drop should be forced
 IF (exists (SELECT * FROM dbo.sysobjects WHERE id = object_id(N'[' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + ']') and OBJECTPROPERTY(id, N'IsUserTable') = 1) and @DropAuditTable = 1)
 BEGIN
  PRINT 'Dropping audit table [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + ']'
  set @sql = 'drop table [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + ']'
  print @sql
  EXEC (@sql)
 END
 
 -- Declare cursor to loop over columns
 DECLARE TableColumns CURSOR Read_Only
 FOR SELECT b.name, c.name as TypeName, b.length, b.isnullable, b.collation, b.xprec, b.xscale
  FROM sysobjects a 
  inner join syscolumns b on a.id = b.id 
  inner join systypes c on b.xtype = c.xtype and c.name <> 'sysname' 
  WHERE a.id = object_id(N'[' + @Owner + '].[' + @TableName + ']') 
  and OBJECTPROPERTY(a.id, N'IsUserTable') = 1 
  ORDER BY b.colId
 
 OPEN TableColumns
 
 -- Declare temp variable to fetch records into
 DECLARE @ColumnName varchar(128)
 DECLARE @ColumnType varchar(128)
 DECLARE @ColumnLength smallint
 DECLARE @ColumnNullable int
 DECLARE @ColumnCollation sysname
 DECLARE @ColumnPrecision tinyint
 DECLARE @ColumnScale tinyint
 
 -- Declare variable to build statements
 DECLARE @CreateStatement varchar(8000)
 DECLARE @ListOfFields varchar(2000)
 SET @ListOfFields = ''
 
 declare @AuditTableExists int
 declare @existsParams nvarchar(4000)
 set @existsParams='@Exists int output'
 set @sql = 'if exists (SELECT * FROM [' + @AuditDatabaseName + '].information_schema.tables  WHERE table_schema=N''' + @Owner + ''' and table_name=''' + @TableName + @AuditNameExtension + ''') set @Exists=1 else set @Exists=0'
 -- Check if audit table exists
 print @sql
 Exec sp_executesql  @sql, @existsParams, @Exists=@AuditTableExists output
 print @AuditTableExists
 IF @AuditTableExists = 1
 BEGIN
  -- AuditTable exists, update needed
  PRINT 'Table already exists. Only triggers will be updated.'
 
  FETCH Next FROM TableColumns
  INTO @ColumnName, @ColumnType, @ColumnLength, @ColumnNullable, @ColumnCollation, @ColumnPrecision, @ColumnScale
  
  WHILE @@FETCH_STATUS = 0
  BEGIN
   IF (@ColumnType <> 'text' and @ColumnType <> 'ntext' and @ColumnType <> 'image' and @ColumnType <> 'timestamp')
   BEGIN
    SET @ListOfFields = @ListOfFields + @ColumnName + ','
   END
 
   FETCH Next FROM TableColumns
   INTO @ColumnName, @ColumnType, @ColumnLength, @ColumnNullable, @ColumnCollation, @ColumnPrecision, @ColumnScale
 
  END
 END
 ELSE
 BEGIN
  -- AuditTable does not exist, create new

  -- Start of create table
  SET @CreateStatement = 'CREATE TABLE [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + '] ('
  SET @CreateStatement = @CreateStatement + '[AuditId] [bigint] IDENTITY (1, 1) NOT NULL,'
 
  FETCH Next FROM TableColumns
  INTO @ColumnName, @ColumnType, @ColumnLength, @ColumnNullable, @ColumnCollation, @ColumnPrecision, @ColumnScale
  
  WHILE @@FETCH_STATUS = 0
  BEGIN
   IF (@ColumnType <> 'text' and @ColumnType <> 'ntext' and @ColumnType <> 'image' and @ColumnType <> 'timestamp')
   BEGIN
    SET @ListOfFields = @ListOfFields + @ColumnName + ','
  
    SET @CreateStatement = @CreateStatement + '[' + @ColumnName + '] [' + @ColumnType + '] '
    
    IF @ColumnType in ('binary', 'char', 'nchar', 'nvarchar', 'varbinary', 'varchar')
    BEGIN
     IF (@ColumnLength = -1)
      Set @CreateStatement = @CreateStatement + '(max) '   
     ELSE
      SET @CreateStatement = @CreateStatement + '(' + cast(@ColumnLength as varchar(10)) + ') '   
    END
  
    IF @ColumnType in ('decimal', 'numeric')
     SET @CreateStatement = @CreateStatement + '(' + cast(@ColumnPrecision as varchar(10)) + ',' + cast(@ColumnScale as varchar(10)) + ') '   
  
    IF @ColumnType in ('char', 'nchar', 'nvarchar', 'varchar', 'text', 'ntext')
     SET @CreateStatement = @CreateStatement + 'COLLATE ' + @ColumnCollation + ' '
  
    IF @ColumnNullable = 0
     SET @CreateStatement = @CreateStatement + 'NOT '   
  
    SET @CreateStatement = @CreateStatement + 'NULL, '   
   END
 
   FETCH Next FROM TableColumns
   INTO @ColumnName, @ColumnType, @ColumnLength, @ColumnNullable, @ColumnCollation, @ColumnPrecision, @ColumnScale
  END
  
  -- Add audit trail columns
  SET @CreateStatement = @CreateStatement + '[AuditAction] [char] (1) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,'
  SET @CreateStatement = @CreateStatement + '[AuditDate] [datetime] NOT NULL ,'
  SET @CreateStatement = @CreateStatement + '[AuditUser] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,'
  SET @CreateStatement = @CreateStatement + '[AuditApp] [varchar](128) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,' 
  SET @CreateStatement = @CreateStatement + '[AuditContext] [varchar](128) COLLATE SQL_Latin1_General_CP1_CI_AS NULL)' 
 
  -- Create audit table
  PRINT 'Creating audit table [' + @Owner + '].[' + @TableName + @AuditNameExtension + ']'
  print @CreateStatement
  EXEC (@CreateStatement)
 
  -- Set primary key and default values
  SET @CreateStatement = 'ALTER TABLE [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + '] ADD '
  SET @CreateStatement = @CreateStatement + 'CONSTRAINT [DF_' + @TableName + @AuditNameExtension + '_AuditDate] DEFAULT (getdate()) FOR [AuditDate],'
  SET @CreateStatement = @CreateStatement + 'CONSTRAINT [DF_' + @TableName + @AuditNameExtension + '_AuditUser] DEFAULT (suser_sname()) FOR [AuditUser],CONSTRAINT [PK_' + @TableName + @AuditNameExtension + '] PRIMARY KEY  CLUSTERED '
  SET @CreateStatement = @CreateStatement + '([AuditId])  ON [PRIMARY], '
  SET @CreateStatement = @CreateStatement + 'CONSTRAINT [DF_' + @TableName + @AuditNameExtension + '_AuditApp]  DEFAULT (''App=('' + rtrim(isnull(app_name(),'''')) + '') '') for [AuditApp],'
  SET @CreateStatement = @CreateStatement + 'CONSTRAINT [DF_' + @TableName + @AuditNameExtension + '_AuditContext]  DEFAULT (CAST(CONTEXT_INFO() AS VARCHAR)) for [AuditContext]'
 
  EXEC (@CreateStatement)
 
 END
 
 CLOSE TableColumns
 DEALLOCATE TableColumns
 
 /* Drop Triggers, if they exist */
 PRINT 'Dropping triggers'
 
 
 set @sql = '
 IF exists (SELECT * FROM dbo.sysobjects WHERE id = object_id(''[' + @Owner + '].[tr_' + @TableName + '_Insert]'') and OBJECTPROPERTY(id, ''IsTrigger'') = 1) 
  EXEC (''drop trigger [' + @Owner + '].[tr_' + @TableName + '_Insert]'')
 
 IF exists (SELECT * FROM dbo.sysobjects WHERE id = object_id(''[' + @Owner + '].[tr_' + @TableName + '_Update]'') and OBJECTPROPERTY(id, ''IsTrigger'') = 1) 
  EXEC (''drop trigger [' + @Owner + '].[tr_' + @TableName + '_Update]'')
 
 IF exists (SELECT * FROM dbo.sysobjects WHERE id = object_id(''[' + @Owner + '].[tr_' + @TableName + '_Delete]'') and OBJECTPROPERTY(id, ''IsTrigger'') = 1) 
  EXEC (''drop trigger [' + @Owner + '].[tr_' + @TableName + '_Delete]'')'
 
 
 exec (@sql)
 
 /* Create triggers */
 PRINT 'Creating triggers' 
 set @sql = 'CREATE TRIGGER tr_' + @TableName + '_Insert ON [' + @Owner + '].[' + @TableName + '] FOR INSERT AS INSERT INTO ['+ @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + '](' +  @ListOfFields + 'AuditAction) SELECT ' + @ListOfFields + '''I'' FROM Inserted'
 print @sql
 EXEC (@sql)
 
 set @sql = 'CREATE TRIGGER tr_' + @TableName + '_Update ON [' + @Owner + '].[' + @TableName + '] FOR UPDATE AS INSERT INTO [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + '](' +  @ListOfFields + 'AuditAction) SELECT ' + @ListOfFields + '''U'' FROM Inserted'
 print @sql
 EXEC (@sql)
 
 set @sql = 'CREATE TRIGGER tr_' + @TableName + '_Delete ON [' + @Owner + '].[' + @TableName + '] FOR DELETE AS INSERT INTO [' + @AuditDatabaseName + '].[' + @Owner + '].[' + @TableName + @AuditNameExtension + '](' +  @ListOfFields + 'AuditAction) SELECT ' + @ListOfFields + '''D'' FROM Deleted'
 print @sql
 EXEC (@sql)
 
END

Use the comments for any issues, questions and of course comments :)

Happy auditing!

Luuk

Sunday, February 19, 2012

Export and import a large set of resx resources

For a project at work we needed to export a lot of resource tables to excel and let an external party validate, correct and complement them. For this we used simple excel sheets. But then the application grows larger, so do the resource tables. With more than 13 languages this becomes a huge pain in the ***. To overcome this we've build a Resource Extractor which uses NPOI to export and import the data.

Currently only the Export functions are included. Once the import is finished I'll write another more complete post.


If you have even more translations to do you can also use dedicated websites so external parties can help you even better with translating. The one I've seen which looks very impressive is amanuens. If you have any experience with them or any other, please let us know in the comments.

The source is available at GitHub https://github.com/luuksommers/resourceextractor