Automate your DB deployment with DbUp

Databases and applications should coexist in the same deployment pipeline. They should be versioned and packaged in the same way and they should share a single development lifecycle. This will enable the team to treat DB deployment like any other software deployment.  To achieve this with SQLServer you can use DbUp, it’s a simple but very effective migration tool.  Have a look at they getting started doc: https://dbup.readthedocs.io/en/latest/

 

The monolith database as a primary cause of slow delivery

Large monolitic applications are characterized by big bottlenecks that slows down the entire process.  The dependencies causes a significant overhead on your deployment throughput. Cycle times are longer because of the slower builds and automated and manual tests required to validate the entire app. This creates a lot of friction in the pipeline. Dependencies brings complexity what increases the development time. Complex code bases have more opportunities for bugs what causes many waste. The dependencies also lead to less opportunities for parallelizing the work. To be able to work in group, developers need to create branches so that developers can work in isolation but this demand also to merge more often what also demand a considerable effort. 

For me bottleneck number one is Manual Testing! Manual Testing is a tedious process when the test cycles are repeated frequently. Therefore many teams adopt test automation but before they can do test automation they first need to create test data. Only when teams have adequate test data they can run automated tests, and can create that data on demand, they see better IT performance, lower change failure rates, and lower levels of deployment pain and rework. The problem with large monolitic applications is that they also go with big databases and how bigger the database how more difficult it is to manage. When the test data is difficult to setup or reason about then it become difficult to do any meaningful test automation.

Conclusion, if your database is a monolith your delivery process we’ll suffer from bottlenecks caused by lack of test automation and also by communication/coordination overhead even if you have a service oriented or microservice architecture on top of it. 

Continous Delivery as a paradigm shift

In the classical view of software delivery, you need to make a trade-off between quality and speed. The traditional IT delivery process paradigm postulate that building quality software will demand longer to deliver. You trade quality with getting things done rapidly. Now, studies on teams that have adopted Continuous Delivery (CD) demonstrate exactly the opposite. Teams able to deploy by the push of a button not only deliver faster but they deliver better quality in a more reliable way.

 

Being able to push changes fast and reliably to production changes the rule of the game and transforms many aspects of the IT delivery process.  It reflects on aspects like:

Increased Availability & Manageability

Because you can push changes a lot faster you can make smaller changes with less downtime you don’t need to come in the middle of the night or weekends to deploy the system.  This means less overwork and stress for the team’s members. It also frees up a considerable amount of time as teams don’t need to participate at long lasting release planning's sessions to coordinate huge deployments.

Because you know you can push a change fast you’re also able to remediate faster and you don’t need to take account of being able to reverse a giant change. In place, you can adopt release strategies where you always advance forwards. In place of reverting a change you can be confident that you can fix any problem fast. This simplify the release process and lower the investment needed in process, infrastructure and tools to be able to revert back a giant change.

Reports also demonstrates that teams practicing CD sees their number of big incidents diminishing drastically. Apart from the better intrinsic quality that CD brings this is also due to the automation that CD demands.  The automated deployment pipeline eliminates errors that might result from manual configuration and error-prone practices. 

Increased efficiency and cycle time
Because you make smaller changes your tester and analyst can work in a constant flow of work.  They don’t need to test everything in a great batch at the end of the release but can continuously work in at constant pace together with the developers.

Because bugs are also detected sooner they are cheaper to fix.  When practicing continuous delivery automated tests are run immediately after each check-in what detects many bugs immediately.  The developer can get immediate feedback even before he starts working on another task and so he knows where to search for the bug.  It’s proved by many studies that bugs found before or immediately after the check-in are orders in magnitude cheaper to fix (sometimes more than 100 times).   *https://www.researchgate.net/publication/255965523_Integrating_Software_Assurance_into_the_Software_Development_Life_Cycle_SDLC

Improved testability

To be able to deploy small changes independently the systems needs to be decoupled.  This decoupling enables also to test the system more easily in isolation.  Most of the changes can be tested without deploying to an integration environment. This lowers the number of environment needed and allows for a more rapid feedback. 

Improved intrinsic product quality

Because the tremendous acceleration of time to market of a change, from once every 3 months to many times a day, product owners can experiment a lot faster enabling new product evolution strategies based on experimentation and continuous improvement.  The book Lean Enterprises describes in detail how modern software companies takes advantages of continuous delivery to design products based on experimentation.

Improved Customer Satisfaction

CD brings confidence to the customers and other stakeholders on the delivery of their new features.  CD enables to deliver new features at a constant and predictable pace and increase the visibility of the delivery process.  By practicing feature toggles and decoupling the deployment from the actual release of the feature it enables to deploy a part or a complete new capability to early adopters.  

 

Because organizations are more effective they can spend more time on value adding activities.  This creates a virtuous circle of continuous improvements that brings a decisive competitive advantage.  Continuous Delivery is becoming the norm in the enterprise. IT organization that still stick to the old delivery model will be forced to change or will simply disappear.  Following the 2015 State of DevOps practices study, the companies who their IT team’s practices continuous delivery has 50 % higher market capitalization growth over the 3 last years than the other and Gartner predicts that by 2020 50% of CIO’s that have not transformed their capabilities will be displaced.  

Continous delivery can't be bought in a box!

Continuous delivery is the ability to get changes into production safely and quickly in a sustainable way. 
It’s not only about deploying code in production but it’s about bringing all type of changes that helps to deliver value faster. This includes all types of IT infrastructure changes, software configuration changes, deploying data and schema migrations, …
It is a capability an enterprise need to develop and learn. Because it’s based on new paradigms that demands deep organizational changes, it’s not something an enterprise can acquire in a box but it need to develop it through a change journey.

It took Amazon or Facebook years to change their organization and architecture so that they could be sustainable by delivering faster. It can demand tremendous efforts and changes to the culture and architecture. 
Like any big change in an organization it’s not linear, how you will achieve the goal is largely unpredictable. Therefore like any change in the enterprise, it need to be carefully managed to ensure that current operations are not adversely affected.

How to design your SPA to work with Angular CLI and DoNet Core

When building new apps, you need both client and server-side pieces.  This article focusses on how to structure a SPA that is made out of angular for the client and dotnet core for the server.

There are numerous questions I needed to answers when mixing these technologies.

  • Should I host my angular app inside an MVC View or just serve the angular app from static files.
  • Should I host the angular app inside the same application that serve the Web-Api or serve it from a separate site?
  • How to setup a productive development environment?
  • How should I integrate angular-cli together with a ASP.NET Core app?
  • Should I use Vs.Code or VisualStudio ?
  • How to package and deploy the application for on prem or in the cloud?

Of course, the answer to all these questions is always the same: it depends! After experimenting with several options, I finally decided that I should go for the simplest solution that could possibly work and started with just serving static files inside a Web-Api application.  Yet, during development, what I found the most productive, is to completely separate the Angular app from the Web-Api solution. I like that my Angular and the ASP.NET Web-Api apps live separate folders beside each other. It’s only at build time that Msbuild assemble the parts by calling “ng build”.  To do that I needed to configured angular-cli to output the build in the \wwwroot folder of the ASP.NET app.

Concerning the IDE, what I found the most productivity was to develop my angular app using Vs.Code.  Thus far, for the Web-Api part I tend to prefer VisualStudio and Resharper. So, I optimized my Angular app for working with Vs.Code and my Web-Api project for VisualStudio.  During development I usually run the backend and launch the Webpack dev server through an “npm start” command.  At publish everything is nicely packaged together inside a ASP.NET app, so deploying to Azure can simply be done by using the VisualStudio publish wizard.  

On my github project core-angular-example, you can download the solution template I use for making SPA’s with Angular-cli and DotNet Core.

Here I explain the steps you need to follow to setup the basis of such an app:

0) First you need to install:

1) From a powershell console:

mkdir angular-dotnetcore
cd angular-dotnetcore
mkdir web-api
cd web-api
dotnet new webapi
cd ..
ng new angular-app

2) Open VisualStudio code and open the angular-dotnetcore folder

3) Disable TypeScript compilation
Since we will eventually have TypeScript files in our project, we should also disable any TypeScript compilation errors. Open the web-api.csproj file and modify the top PropertyGroup:

 

  <PropertyGroup>

    <TargetFramework>netcoreapp2.0</TargetFramework>

    <TypeScriptCompileBlocked>true</TypeScriptCompileBlocked>

    <TypeScriptToolsVersion>Latest</TypeScriptToolsVersion>

    <IsPackable>false</IsPackable>

  </PropertyGroup>

 

4) Configure Startup.cs
As our web-api app will also serve the static files we need to configure the Stratup.cs file as follow:

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }

    app.UseDefaultFiles();
    app.UseStaticFiles();

    app.Use(async (context, next) => {
        await next();
        if (context.Response.StatusCode == 404 &&
            !Path.HasExtension(context.Request.Path.Value) &&
            !context.Request.Path.Value.StartsWith("/api/")) {
                context.Request.Path = "/index.html";
                await next();
            }
    });

    // Configure application for usage as API
    // with deafult route '/api/[Controller]'
    app.UseMvcWithDefaultRoute();

    // Serve static files
    app.UseStaticFiles();
}

 

 

 

5) Configure angular-cli to output the build inside the wwroot of the web-api app =>
edit the .angular-cli.json file and change the OutDir parameter:

 
"apps": [
    {
      "root": "src",
      "outDir": "../web-api/wwwroot",

Now when running ng build inside the angular-app angular-cli should output the result inside the wwroot folder of the web-api app.

6) Modify the angular app to consume data from your web-api
Edit the following files under /angular-app/src/app file=>

App.module.ts:

import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';

import { AppComponent } from './app.component';
import { HttpModule } from '@angular/http';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    HttpModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

Modify app.component.ts:

import { Component, OnInit } from '@angular/core';
import { Http } from '@angular/http'
@Component({
   selector: 'app-root',
   templateUrl: './app.component.html',
   styleUrls: ['./app.component.css']
})
export class AppComponent implements OnInit {
   constructor(private _httpService: Http) { }
   title = "DotNet Core Angular Example";
   apiValues: string[] = [];
   ngOnInit() {
      this._httpService.get('/api/values').subscribe(values => {
         this.apiValues = values.json() as string[];
      });
   }
}

Edit app.component.html and add this as last lines of the html file:

  <li>
    <h2>Values from ValuesController</h2>
    <p *ngFor="let value of apiValues">{{value}}</p>
  </li>
</ul>

7) Redirect api calls to your DotNet core app

To be able to redirect api calls from your angular app when running  in the webpack dev server you should create a proxy.conf.json file on the root of your angular app.  Here my dotnet webapi runs on port 5000:

{
    "/api": {
      "target": "http://localhost:5000/",
      "secure": false
    }
}

You can already test your app:

  • launch the web-api app with

dotnet run

  • Launch the angular app and pass the proxy configuration:

ng serve --proxy-config proxy.conf.json

8) To be able to use npm script commands edit the package.json of the angular-app and edit the script section:

"scripts": {
    "ng": "ng",
    "start": "ng serve --proxy-config proxy.conf.json",
    "build": "ng build",
    "build-prd": "ng build --prod --env=prod",

9) To be able to invoke these commands from you web-api, add a package.json inside the root of the web-api app containing.  Here we declare the same commands but first cd to the angular-app:

{
    "name": "core-angular-example",
    "version": "0.0.0",
    "license": "MIT",
    "scripts": {
        "install": "cd ../angular-app & npm install",
        "start": "cd ../angular-app & npm start",
        "build": "cd ../angular-app & npm run-script build",
        "build-prd": "cd ../angular-app & npm run-script build-prd"
    }
}

10) Finally, we configure our web-api MsBuild script to invoke the ng build script when running a release build. Therefore, we add the following targets to it:

<Target Name="EnsureNode">
  <Exec Command="node --version" ContinueOnError="true">
    <Output TaskParameter="ExitCode" PropertyName="ErrorCode" />
  </Exec>
  <Error Condition="'$(ErrorCode)' != '0'" Text="Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE." />
</Target>

<Target Name="ReleaseRunNgBuild" BeforeTargets="Build" Condition=" '$(Configuration)' == 'Release' ">
  <CallTarget Targets="EnsureNode" />
  <Message Importance="high" Text="Install packages..." />
  <Exec Command="npm install" />
  <Message Importance="high" Text="Performing ng build for prd build..." />
  <Exec Command="npm run-script build-prd" />
</Target>

Now when you launch a Release build everything will be build together. 

You can get a package containing all your app by using the publish command.

If you want to publish your site to azure simply use the VisualStudio publish wizard or the Azure Powershell Publish commands.

How to design your SPA to work with Angular CLI and DoNet Core

When building new apps, you need both client and server-side pieces.  This article focusses on how to structure a SPA that is made out of angular for the client and dotnet core for the server.

There are numerous questions I needed to answers when mixing these technologies.

  • Should I host my angular app inside an MVC View or just serve the angular app from static files.
  • Should I host the angular app inside the same application that serve the Web-Api or serve it from a separate site?
  • How to setup a productive development environment?
  • How should I integrate angular-cli together with a ASP.NET Core app?
  • Should I use Vs.Code or VisualStudio ?
  • How to package and deploy the application for on prem or in the cloud?

Of course, the answer to all these questions is always the same: it depends! After experimenting with several options, I finally decided that I should go for the simplest solution that could possibly work and started with just serving static files inside a Web-Api application.  Yet, during development, what I found the most productive, is to completely separate the Angular app from the Web-Api solution. I like that my Angular and the ASP.NET Web-Api apps live separate folders beside each other. It’s only at build time that Msbuild assemble the parts by calling “ng build”.  To do that I needed to configured angular-cli to output the build in the \wwwroot folder of the ASP.NET app.

Concerning the IDE, what I found the most productivity was to develop my angular app using Vs.Code.  Thus far, for the Web-Api part I tend to prefer VisualStudio and Resharper. So, I optimized my Angular app for working with Vs.Code and my Web-Api project for VisualStudio.  During development I usually run the backend and launch the Webpack dev server through an “npm start” command.  At publish everything is nicely packaged together inside a ASP.NET app, so deploying to Azure can simply be done by using the VisualStudio publish wizard.  

On my github project core-angular-example, you can download the solution template I use for making SPA’s with Angular-cli and DotNet Core.

Here I explain the steps you need to follow to setup the basis of such an app:

0) First you need to install:

1) From a powershell console:

mkdir angular-dotnetcore
cd angular-dotnetcore
mkdir web-api
cd web-api
dotnet new webapi
cd ..
ng new angular-app

2) Open VisualStudio code and open the angular-dotnetcore folder

 

Angular2 vs React from the perspective of a .Net fullstack developer

FightClub

Intro

Currently we had a lot of discussion on which web framework we should invest: Angular2 or React. Therefore I decided to experiment with both and make my own meaning. 

Coming from Angular1 and loving it, I can’t pretend I’m completely objective. Nevertheless being a little frustrated with the breaking changes and late arrival of Angular2 and hearing a lot about the goodness of React, I wanted to give it a try and compare both frameworks with an open mind-set.  

I took the perspective of an Enterprise .Net developer.  In this comparison I value support for tooling that just work over purity/beauty of the framework.  I also tested the integration with Visual Studio 2015 and Visual Studio Code.

In our environment applications have a lifecycle that can easily stretch over 10 years. In this context be able to support big projects and keep maintaining them over time is key. This is why I took Typescript as a given.  Coming from a .Net background and having built all my Angular apps with the help of Typescript I don’t wanted to get back to pure JavaScript. I love the static type checking and all the tooling support, code completion and refactoring typescript brings to the table.  So the applications I wrote to compare both frameworks used Typescript and the ease of programming with Typescript is for me important.

Overview

React

React is a declarative Javascript UI library, created by Facebook that handles a very specific set of concerns, composing and packaging your UI in components and rendering these components. What could seem odd in the beginning is that React doesn’t separate the HTML from the JavaScript.  Frankly I got used to it and learned to appreciate the advantages of this. You can still separate the logic from the presentation by creating components that only render Html and Components that handles the logic  (named “Controller Views” components). 
Here is a simple example of a React component:

import * as React from "react";

export class HelloWorld extends React.Component<{}, {}> {
    render() {
         return (
            <div>

                <p>Hello world from react!</p>

            </div>
        );
    }
}

The render() method is the core method of the React component.  React implements a virtual DOM, this is an internal representation of the DOM.  Using his own representation of the real DOM and only rendering what was changed since last time render() was called. 

A react component is typically written in a JSX file.  JSX enables to mix the HTML tags inside the JavaScript part.  To transform the JSX at runtime or compile time you typically need to use Babel. Now TypeScript provide support for JSX, this enable us to use the TypeScript compiler instead of Babel. Adding the compiler option: “jsx”:”react” in your tsconfig.json file will enable support for the the TypeScript compiler.  When this is done you can use the “tsx” extensions in order to support “JSX” with typescript.

I found that VisualStudio 2015 and VS Code provided a good support for the TSX format.  It differentiates the HTML and TypeScript part quiet well. With TSX you even get auto completion in the HTML parts.
 

To illustrate other key concepts of React we need a more sophisticated example:

 

Notice on line 9, the two type arguments we pass to declare our AuthorComponent class.  The first is the “prop” the other is the “state”.  These two concepts should not be confused.  “state” is an internal concern of the component, it holds the internal data (model) of the component and can change, and every time it changes the virtual DOM is adapted.  “props” are passed to the component and are not meant to be changed.  An illustration of the use of “props” is with “Controller Views Components”, these are parent components that host child components. “props” are typically used to pass the component properties:

Here author, onChange, onSave, errors are “props” of the AuthorForm component.


In contrast with Angular, React is only a UI library; therefore it’s usually used with other libraries like Redux.  Redux help us to manage the application data flows.  You don’t absolutely need Redux but usually when building large scaled Enterprise applications it’s something you’ll want to add to your React apps.  In fact you can even use Redux with Angular but the unidirectional flow of React lent itself very well to the Redux model.  Redux is based on the original Facebook Flux library. It help to manage the application State. I found that managing state in a React application without Flux or Redux demand a considerable effort. Nevertheless these frameworks bring also a great amount of concepts.  Explaining these is outside the scope of this article.

Angular2

Although Angular2 comes with a lot of breaking changes and you can’t upgrade directly from Angular1,  Angular2 still feels like an evolution of Angular1 and I felt that’s the big changes that where introduced where relatively easy to learn coming from an Angular1 background.

For me coming from the .Net/MVVM world, Angular always felt familiar and I appreciate the similarities with the WPF data binding model.  Nevertheless I still have troubles with the new binding syntax that is quite different of the used in Angular1. 

Angular advocates the separation of HTML and Code.  Although these can now be in the same file you don’t mix code & HTML in Angular.   This’s usually a good practice especially when working with web designers.  The downside of using a specific data-binding syntax  is that you don’t get the code highlighting & code completion you get with React.

Angular2 is a framework and a quite complete one.  With Angular you get all in a box.  You can replace many parts but I never really needed to.  From an Enterprise viewpoint, be able to count on a complete but modular toolbox is an advantage.  It helps new hired developers to be productive faster and enforce consistency.

Here is an example of what a “Hello world” Angular2 app looks like.  A small tutorial is available on the site of angular on how to setup a quickstart app.  It took me less than 5 minutes to complete.

After completing this 5 min tutorial you get basically everything you need for client side development including a small web server (BrowserSync) configured to do edit & continue. 

Angular separate the HTML from the code, you can add the Html inside a Component but it’s still in a separate section. To be able to add view logic inside the HTML you need to use the Angular2 template syntax. A simple Angular HTML component looks like this:

The *ngFor, {{todo.title}}, (click), [todo] and <todo-detail> elements are all Angular2 template syntax.  The <todo-detail> tag represent a TodoListComponent.
The todolist.component.ts code could be like this:

Here is an example of how to define a two-way databinding works in Angular2:

<input type="text" [(ngModel)]="user.first_name" />

 

Every Angular2 app begins by declaring at least one module.  This is an example of such a module:

 

import { BrowserModule } from '@angular/platform-browser';

import { FormsModule }   from '@angular/forms';

import { HttpModule }    from '@angular/http';

import { Router }        from '@angular/router';

import { AppComponent }             from './app.component';

import { TeamComponent }            from './team/team.component'

import { TeamService }              from './services/team.service'

import { routing }                  from './app.routing';

 
@NgModule({

    imports: [

        BrowserModule,

        FormsModule,

        HttpModule,

        routing

    ],

    declarations: [

        AppComponent,

        TeamComponent,

    ],

    providers: [TeamService],

    bootstrap: [AppComponent]

})

 
export class AppModule {

}

 

My evaluation

Angular2 loves TypeScript

For me the biggest difference between both is that Angular2 is TypeScript native and React just tolerates it. Sure you can use TypeScript with React, I provided some examples how you can do that nevertheless I spend a lot of time on figuring how.  This is especially a problem when you consider that the React examples you find on the web are inconsistent. You can find many ways to do the same but some of them are not TypeScript friendly.  During my experimentation with React I often was forced to use the <<any>> type.  This means that I lost the goodness of working with TypeScript: compile time type checking, autocomplete and refactoring support.

Angular2 feels like home for C# devs

The React documentation provided by facebook is quite good but the Angular2 official documentation is just superb.  Following the quickstart guide I was able to construct a full application in less than 5 minutes.  You can also find a complete guide that covers every topic of the framework.  The Angular2 “Tour of Heroes” is a tutorial where you build a real world application using every important aspect of Angular2. 

Every example in the Angular2 documentation is provided in TypeScript.  For React most documentation is in ES6. On other internet sites I found that most of the examples are still in ES5. Finding an up-to-date tutorial on how to bootstrap a standard React application with TypeScript is not easy. You can find one on the TypeScript site but it’s very basic.  When you compare this to Angular2 where you even have guidance on how to setup VisualStudio you immediately understand that Angular2 is far more welcoming for C# devs as React.  The only guidance I found on how to integrate TypeScript was on the TypeScript site. What I learned here is that usually you need to figure out by yourself how to use TypeScript with React as you don’t find a lot on the internet. So during my React experimental project I’ve wasted many, many hours in converting Javascript examples to TypeScript. I had some problems with inconsistencies between type definitions; sometimes I needed to tweak this myself to get my project compiled.  I didn’t have any of these problems using Angular2, the type definitions where always up to date and all provided libraries worked well together.   

I love mixing Html inside JavaScript

When working with React in VS.Code and VisualStudio, I was first disappointed as these editors didn’t support the JSX syntax correctly.  All this changed when I converted my first code to TSX (the TypeScript equivalent of JSX).  Here React really shined as I got type checking & code completion even inside my TSX templates.   Mixing typescript inside the HTML was for me a little revelation.  I still think web designers will not really like this but for me other frameworks should copy this great idea.  You don’t need to learn another programming or templating syntax. You can use the same programming language everywhere. When you use TypeScript you even get static type checking in your templates what considerably diminish the number of mistakes you can make. Those who have already used Angular will certainly see the benefit of this.  With Angular I can’t count the number of times I got a blank page or things missing on my page because I misspelled some property in the template or used the wrong templating syntax.  

ErrorMessages feels a lot better

Another great aspect of React is the explicit error messages it provides.     

     

In Angular1 you get usually at best a blank screen with completely useless error messages, now I found the error messages had improved tremendously in Angular2. It usually returns explicit messages with descriptions on how you can solve the problem. 

Many ways to do the same thing

A pain point concerning React is the many ways to do the same thing. For example I found the following ways to create a React component: using the ES5 Create Class Component, using the ES6 class component, using stateless components, with Object.Create, with Mixins, creating Parasitic Components and with the StampIt library.    Also a primary concern for me is the high churn rate of React, currently React is at version 15.  This is a high number for a framework that is only a couple of years old.

Conclusion

For me the conclusion is that if you are a C# enterprise developer who wants to get all the goodness Typescript brings to the table, Angular2 is the obvious choice.  React is a beautiful library that enforces good design but it doesn’t embrace Typescript as Angular2 does.  React has certainly some advantages compared to Angular2 like using JavaScript as templating syntax but Angular2 is what an enterprise developer wants. It’s complete, opinionated and his Typescript native support makes it more crafted for large scale applications.  Angular2 two-way binding also result usually in a higher productivity rate and less code to maintain when it comes to building business applications.

 

Building TypeScript with Gulp

In the preceding post I show how you can build your TypeScript with Tsc, you can also simply use Visual Studio and use the option: “Compile TypeScipt on Build” in the Build properties of your ASP.NET Core project. While these are valid options I generally prefer to use Gulp to setup my own workflow. 

In this post I’ll explain how you can use Gulp to setup and fine tune your Typescript build.

Pull Gulp npm packages

First will need to use npm to pull down some gulp packages that will help us in setting up our Typescript build.

The npm packages we need are:

"gulp-rimraf": "^0.2.0",

"gulp-sourcemaps": "^1.6.0",

"gulp-typescript": "^2.13.4",

"gulp-uglify": "1.2.0",

"rimraf": "2.2.8"

 

To install these packages you can copy paste the list here above right under the “devDependencies” of the package.json file but I prefer to use npm.

Navigate to the root of your project and use the command:

c:\Dev\MyClub\web>npm install [packagename] --save-dev

The --save-dev option is used to update our packages.json and put the new dependencies under the “devDependencies”.

Create the Gulp Tasks

Here we’ll use gulp-typescript to compile automate the typescript build. The documentation can be found here. Open the gulpfile.js and modify as follows:

  • Require the following packages:
  • Add following path variables
  • As we’ll use the tsconfig.json file to set the build properties we’ll create a project:
  • Write the typescript build task
  • Write the clean tasks
  • If you use VS2015 you can bind the gulp tasks with your VS build. The clean tasks should run before our build and after our build we should run first the ‘tsbuilld’ and then the ‘min’ task.

 

 

  • If you want that the ts build is triggered when a ts file changes write a watch task:

 

  • If needed modify your angular script links. As the predefined min:js gulp task mimifies all js files we can choose to use the mimified app.js file in production.

 

Now every time you compile your project or save a typescript file the ts compiler should build your javascript files.

Build your Arduino Robot and program it with Javascript

In this post I explain step by step how I build an Arduino based Robot. This can be a handy if you want to build your own. I explain which design choices I made, which frameworks and technologies I used and how I program and configured the robot. Here you can find a video of the end result.

What is needed?

The robot is based on a RoverV2 + Bluetooth Arduino robotic kit sold by Robotshop.  This kit contains:

  • A compatible Arduino board including:
    • Dual H-bridge and onboard voltage regulator
    • An onboard charger
  • A Bluetooth module
  • An aluminum frame
  • Tamiya Twin-Motor Gear Box
  • Tamiya Track and Wheel Set
  • Lithium Polymer Battery Cell – 3.7V 1000mAh

robotshop

The kit is complete and well documented but if you want to go hard core and start with a classic Arduino board, you can obtain the same result with following components:

  • An Arduino compatible board
  • An Arduino motor shield
  • A pair of two 5v motors and 4 Wheels & frame
  • A Bluetooth module
  • 4xAA battery & holder

The goal of this article is to set the basic building blocks to build a robotic platform that you can extend to finally build an autonomous robot.  Therefore, to enable the robot to sense his environment we extend the base platform with:

3 x ultrasonic sensors HC-SR04

ultrasonic

1x Accelerometer MMA7361.

accelerometer

1x Compass HMC5883L

compass

You can find all these components, including the wheels & frames in shops selling Arduino compatible boards online like SainSmart or Robotshop.

 

Design

For the design I choosed to remotely control the robot from a server (a laptop or pc).  This design overcomes the limited processing & storage of the Arduino board but it comes at the expense of autonomy. My robot will not be able to evolve without be connected by Bluetooth to his server.  This is a concession I’ve to make but it provides me quasi infinitely processing power and makes my robot accessible through the internet.

The Arduino board plays the role of central nervous system and the PC is the brain. Another advantage that offers this design is that the robot can be controlled from anywhere in the world as it is exposed through http by a node.js application.

On the Arduino board runs a small program named “Firmata”.  This program enables you to control the Arduino board remotely through the Firmata protocol.  You’ll send commands from the server to the robot through a serial connection.  On the server side you’ll program your robot with Javascript using the johnny-five node library.  This library provides an abstraction over the Firmata protocol and enable us to interact with the Arduino board and a lot of the Arduinos compatible electronic devices like sensors and motors.

The robot can be remotely controlled and followed through a web interface dashboard.  The dashboard is a single page application built with angular.js   .  The html, images, css and javascript resources are served by express.js.  Express.js is the default web application framework for node.  Express.js is also used to expose a rest based API.   Through this REST interface, the radar and accelerometer data can be retrieved and commands can be send to the motor.  This allow you to program and control the robot via any device using the programming language of your choice.

To enable real time communication in both directions between the server and the robot I’ve used socket.io.

design

 

 Assemble and configure your robot.

To assemble the DFRobotshop RoverV2 you can follow this video.  As I’m a Lego fan and that with two little boys in house I’ve a lot of little bricks, I’d no difficulties to find all pieces I needed to build a Lego frame where I could mount the ultrasonic sensors, the accelerometer and the compass on it. Here you can find a picture of the end result.

robot

It’s up to you to decide to copy the design or make your own as long as you use the same core components and follow the schematics provided here beneath the code will be compatible with your robot.

Scematics

Here you can find the schematics for your robot:

Johnny5-Radar_bb

 

Accelerometer

As the accelerometer MMA7361 can vary slightly between manufacturers and models I provide an alternative schematic only for the accelerometer:

schematicsaccel

Configure your Robot

Prerequisites:

Before you start make sure you’ve setup the following tools on your box:

– Git : https://git-scm.com/

– Node 0.10.40 for x86 : https://nodejs.org/dist/v0.10.40/

– Python 2.7.x : https://www.python.org/downloads/release/python-2710/

If you’re on windows make sure you’ve configure your PATH correctly for all of these tools, therefore I suggest you use Chocolatey to install all of these components.

 Download the code

Create your working folder:
c:\Dev>mkdir arduino
cd arduino

Clone the repository:
c:\Dev\arduino> git clone https://github.com/geobarteam/johnny5.git

Install the packages:

c:\Dev\arduino\johnny5> npm install

You could receive some warnings here, don’t panic it should still be working fine.

Flashing your Arduino board

Because the onboard bluetooth ship is limited to 9600 baud I’hd to patch the firmate protocol. This patched version can be found on the root folder.

  1. Open the Arduino IDE
  2. Connect your Robot to your PC through the usb cable and temporarly remove the bluetooth module from your Arduino board
  3. In the IDE select the correct com port
  4. Choose, File, open and select the “firmate_pulse_9600B_patch.ino” file located under: ..\johnny5\firmate_pulse_9600B_patch
  5. Push the upload button
  6. Reconnect your bluetooth module

 

Connect to your Arduino board through Bluetooth

You should go to your Bluetooth settings and associate your Bluetooth with a COM port.  On resent versions of windows you should click on the Bluetooth icon, choose “Add Bluetooth devices”.  You should see “Bluetooth_Bee_V3” click on it and enter the secret code: 1234.

You need to know which COM port windows as configured to your outgoing connections to you robot.  Therefore open the Bluetooth settings, select the COM ports tab and note which COM port is associated with your Bluetooth_Bee_V3 outgoing connection.

comports

Configure and start the server

With your preferred IDE modify the variable “comport” in the “app.js” file with the COM port you connected the Arduino with.

On the Johnny5 folder, start the program => “node app.js”

console1

console2

Start your preferred browser on http://localhost:3000 , you should now be able to control your robot through the interface:

dashboard

 

Enjoy!