Arthur Wang's Blog
Follow me on
  • My General Blog
  • Software Development
    • Latest Articles on Software Development
    • Complete Resources for Developers >
      • Tutorial Information for Developers
      • .NET Developer Blogs
      • Developer Journals and Magazines
      • Developer's Tools
      • Database Development
      • ​Developer Conference and Events
  • Tech
    • Latest Articles on Technology Development
  • Health
  • Money
  • Services
    • Modern Website Design
    • Web Maintenance of Existing Websites Service
    • Corporate Business Consulting Service
  • About
  • Contact
  • Art
  • 中文部落格

Taming the Wild Wild Web Development Tools like NPM, Bower, Gulp, Grunt and Node with Visual Studio 2015

5/26/2016

0 Comments

 
Picture
​If you are new to Visual Studio 2015 and wonder why all these external web development tools, such as Bower, NPM, or Grunt, were included in the new Visual Studio, then this article is for you.  There are tons of articles written on how to use specific tools and recommendations on which one is better than others.  In this article, we want to try to describe the ecosystem of these modern web development tools in a big picture and how tools have become complicated over time.  The quick and straightforward examples shown here will help you to get started with these tools and understand the reason behinds.

As the ASP.NET developers, we are used to having .NET Framework version upgrades and additional features like NuGet Package Manager and more whenever a new version of Visual Studio comes out.  But this time is not like that. The familiar folder structure is gone, and the web.config file is gone. The folder structure is quite different. There is even a ghostly “Dependencies” folder which appears in the Visual Studio Solution Explorer but not in the physical folders and there are unfamiliar default files [See figure below]. People even seem to move away from using Web Essentials, the Visual Studio Extension, which was a way to use the external tools outside of Visual Studio.  And even the use of NuGet Package Manager is discouraging, and instead, we are seeing more command line tools flourishing in the web development landscape as it seems like we were going backward to early DOS age after all these years of advances.  Are we going back to square one and what’s going on?
Picture
Showing Solution Explorer in Visual Studio 2015
To understand this, we must look at from the web development ecosystem perspective and cannot just look at Visual Studio itself or thought that Visual Studio just want to add some newer tools like NPM or Node.js into their product. We must recognize first that the software development as a whole is now undergoing a rapid growth phase.  New tools or upgrades are coming out on a weekly or even daily basis.  We have never seen such fast pace in the web development when comparing in the past years.   From staying competitive from a product perspective, Visual Studio needs to acquire new external tools and adopt latest web development trend to modernize its product and maximize the number of users. 

Moving away from NuGet

As the Visual Studio users, we are used to let NuGet manage our dependencies, and there is no doubt that it was a great tool.  However, NuGet is gearing toward for the Microsoft .NET ecosystem, and not all client-side libraries are submitted to the NuGet repository.  The Visual Studio users may cut from the outside world and unable to obtain the latest packages or new technologies.  As a result, Visual Studio needs to align itself to adopt the thriving development community. They do not have the resource to chase the trend and reinvent the wheels.  With the time constraint, Visual Studio still finds ways to integrate these new tools into the web development.
 
There are two popular package managers, Bower and NPM, tend to replace the NuGet tool, at least for the non-.NET component such as JavaScript or CSS.  You just need to choose one or both to manage your dependencies. NPM can be used for both server-side node packages and client-side packages.  Bower is more popular client side package manager..

If you use Visual Studio 2015, NPM, Bower will be installed by default.  And you can manage these tool versions from Tool --> Options --> Project and Solutions --> External Web Tools  
Picture
​This list shown on the right from the Option window tells Visual Studio where to find dependencies starting from top to bottom. So the Visual Studio program looks for node_modules\bin folder from your project first, and if it cannot find it, it will look for the folder under C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\Web Tools\External and so forth.
​If you do not use Visual Studio, then you need to install them yourselves in this order.
1. Install node.js from nodejs.org, and it will also install npm by default
2. Install with a command line to install bower by using npm [npm install bower –g]
3. Install Git by using http://msysgit.github.io/  and use Git to download packages from GitHub.  

​Where are the confusions?

​For Windows users, we are so used to have no-brainer installation in the Windows environment.  Just double click and install. But to install a web development tool without Visual Studio 2015, there are so many steps just to install these tools, and we have not even talked about how to use them yet.  Life is easier in Visual Studio 2015.  Before we talked about how to use them in VS 2015, let’s compare Bower and NPM since they are doing the similar things like package managers, but it really dependent on your needs.  For example, if you need to keep two versions of jQuery in the production, then you need to use NPM, but if you need only one single copy of jQuery, you should use Bower since it just installs the designated version packages that you specified. So initially NPM is used for installing Node.js modules, and Bower.js is leaning toward on the use of managing the front end components like CSS, JS, etc. 

Here is how npm works:
 
There is a registry on the internet (npmjs.org) where tool developers can publish their works, and it is powered by the CouchDB database.  Once you installed the node.js on your local machine, from a command line, you can invoke npm.exe to manage your packages.  In your project folder, you will need a meta file named package.json that instructs npm on what to do.  The npm will go to the npm registry to find your missing packages and bring them into node_modules folder on your local machine. 
Picture
​So in Visual Studio 2015, all you need to do is to open the package.json file, and add the name of the package and its version under the devDependencies section, and as soon as you hit the save button, Visual Studio will act as a proxy to npm to grab those packages for you. 
Picture
Showing the content of package.json used by npm.exe
But if you preferred doing it manually, you can do it in the Package Manager Console inside the Visual Studio.  
Picture

​The Build System

​As you may have notice that each package manager has its own destination folder for the packages that it stores. Bower stores in the bower_components folder and NPM stores in the node_modules folder. In this case, we need to have another build step to copy those packages into our web development or production folder from the package folder.  For example, if we use AngularJS in the development, our AngularJS files may be located in node_modules folder if we use NPM, and we want packages to be located in our wwwroot/js folder.  Grunt and Gulp are the two of the most popular build systems out there to do these tasks. However, NPM can act as a build tool as well if you use its createReadStream function.  Nevertheless, if you need more complex build process, many people will likely choose either Grunt or Gulp since it has a lot of plugins to choose from.  In Visual Studio, these tasks located inside the Grunt or Gulp script can be managed via Task Runner Explorer.  You can do more inside the Task Runner Explorer when you right click on the task.
Picture
Conclusion
 
In this article, we have seen how these little tools such as the npm, grunt, and many others, are quite powerful, and they can do a lot of things for us as developers.  The tools help us from managing our dependencies to our build process, and they could also help us test and do continuous integration if we spend some time to configure them with scripts and a workable deployment pipeline can be built at the end. However, because of these tools live in an open source environment, there is no one to supervise and manage, it has its advantages, and there are some disadvantages as well.  When you need a tool's plugin to do something, you may find that you need more dependencies after dependencies, and sometimes it may add unwanted file size to your project.  There are more tools to fill the gaps of other tools. There are management tools to manage other management tools.  There are fixes and upgrades within the tools, configuration scripts may work for a few months and may not work if you upgrade certain tools.  It is an age of disruptor overboard, and it is quite a chaos if you think about it.  Regardless of the frustration that some developers may have experienced, we need to do our part of finding the right tools by testing and prototyping the workflows that fit our needs.  We should always have backup copies of these tools stored somewhere you can access since the process often assumes that you have an internet access.  On the security issue, we should also be alert to the risk of having such an easy but powerful tool like the npm.  Since node.js was already installed on our side, and this is always a vulnerability of allowing the malicious package to come in without any supervision.  Hopefully, the technology will become more mature and reliable sooner. ​The developers can spend more time on building great software and spend less time on finding the right tools for the right jobs and then figuring out how to use the tools to their best interests.  Perhaps, Visual Studio can tame the modern web development tools in the future version.
0 Comments

Better Collaboration Begins with Breaking the Silo Mentality in the Organization

5/10/2016

0 Comments

 
Picture

Natural Cause of Silo Mentality

​In some respects, the problems of the silo mentality hinder the operation of an organization resemble dental plaque problems where it does not happen overnight, and the buildup gaps between each department grow and eventually form a rigid wall that leads to instability of the organization. Similar to the teeth's condition, plague is not preventable, but if it’s regularly removed, it can prevent gum disease and loss of teeth.  In a corporate environment, it is a natural progression that each department grows to have its goals, priorities, and tools as it matures.  But this maturity often leads to silo mentality when each department becomes more self-centered.  It’s making the decision based on self-interest and not on the wellness of the organization. As a result, the efforts in coordinating to share information and tools be between groups or departments can be discouraging and inefficient.

​The Danger of the Silos

​During working in various corporations for the past years, I've witnessed numerous successful executions and inherent failures.  In many occasions, the lack of awareness of the silo mentality is often the cause of the downfall of the project or the organization and the success often accompanied with effective methods of removing the gap between departments.  It is quite detrimental if you are not aware of this type of problem existed since it can be pervasive in every business process and just like the symptoms of illness, you only see the effects when they come to the surface and unable to find the solution that targets the primary cause of the problem.

​The Blame Game

​The blame game and endless spiral of internal conflicts can lead the entire organization to nowhere and standstill.  I've seen a big organization that could not break its gridlocks and standstill for years and could not make a breakthrough from it.  While we do not like to play the blame game and finger pointing, but the key to having a high-performance team without the silo mentality is held by the top executives in the organization.  Only they can solve the chronic organizational problems and lead everyone to overcome resistance to change and by exerting the power to form a unifying organization that achieving common organizational goals.  

​How to Break the Barriers

​I often encourage youngsters by telling them the reasons to go to schools.  I think that attending school is for your benefits, it is a place to make mistakes and to exercise your brain muscle because you are going to need it when you are in the real world.  No matter what you are majoring in, treat it as a reference point to leap from and not the destination of your lives.  Similarly, I often like to tell my subordinates that working in a job is not just for making a living for yourself, it's about learning more about yourself when you interact with others and creating values for yourself.  It's what and how you do determine yourself and not how much you make.  In a similar fashion, any organization wants to create an efficient and high-performance team should let everyone participate in achieving the common goals together and live in a larger story than our own.  This is a proven concept called Challenge Intelligence, which was discovered in the 1960s.  In 1962, President John F. Kennedy had an opportunity to visit the NASA space center, and noticed a janitor carrying a broom, and asked him, "What are you doing?"  To most people, the role of janitor was just cleaning the building, but his simple answer had helped solve one of the most complex problems in the corporate world and was contributing to making history.  He said, "Well, Mr. President. I'm helping put a man on the moon."  It's so true that no matter how large or small your role, you are contributing to the larger story unfolding within your life.  It's like any parts of your body; they work in synergy. Finding the meaning of the organization is like a spiritual experience or it's like the soul of the body.  Endless internal conflicts and frictions suddenly fade away as it’s been replacing with a higher purpose.  And when your entire team embraced and shared a larger than the personal goal, incredible things happen.
 
Besides having a unified vision, the tasks of steering the ship to a new coordinate and making a real transformation corporate-wide can be quite challenging.  Techniques like the Challenge Maps have been used to identify the critical gaps exist in departments and map out the actual steps needed to move forward.  By having a clear plan and workable deliverable, solving chronic organization problem is a reality when you see the whole organization align to the right direction progressively.  When the direction becomes clearer, meetings become much more productive and working together becomes a top priority of its own.
 
For smaller organizations that do not have their IT resources, another way is to employ IT experts who have expertises in the areas of process improvement and systems thinking and develop joined-up solutions between departments and fill the gaps on a regular basis.  These software developers should have extensive knowledge about business processes and have the ability to lead the direction of improvement, rather than just taking requirements and ideas from the stakeholders directly.  However, just like any other functional units in the organization, as a natural progression, silos will form within the IT team as well between the Development and IT Operations groups. 
 
In the recent years, DevOps, an IT movement or practice, tries to solve this particular issue by creating a culture that partners developers with operations staff together to ensure the software delivered with minimal problems where all members in DevOps are willing to support the idea of sharing and working together.  So it is essentially a practice that aims at collaboration and communication of both software developers and other IT professionals.

​Conclusion

​The causality of the silos started with people, and it is logical to assume that people have the key to resolving the problem as well.  By having a unified vision and working toward a common goal, rather than just for a particular department, they are the early steps of tearing down the silos.  The continuous efforts of managers to invest time in understanding the individuals’ needs and words of encouragement are the real engines that drive everyone to move forward.  Establishing goals and measuring the effectiveness of leadership are also necessary for the department heads. Leading a thriving and productive team from the top management team is critical to keep the momentum going.  A great leadership begins when the individuals understand their bigger roles and leaders become entirely depending on and in trust with the people.  There is nothing stronger than having a laser focus force that enables any organization to accomplish anything it desires.
0 Comments

    Arthur Wang

    @ArthurWangLA
    MCSD App Builder
    MCSD Web Applications
    ​Member of Windows Insider Program & HoloLens Developer Community & Dev Center Insider Program

    Over 17+ years of  experience in web-based software development & management.  Specialized in Microsoft technology with c# language and its architecture design.  MCSD, MCSE, Microsoft Specialist, MCP + Internet, and B.S. from UCLA

    Archives

    August 2018
    March 2018
    January 2017
    December 2016
    May 2016
    April 2016
    March 2016
    February 2016
    April 2014

    Categories

    All
    API
    Arduino
    ASP.NET
    Cognitive
    CSS
    Database
    Deep Learning
    DevOps
    Electronics
    Flexbox
    HTML5
    IoT
    Katana
    Machine Learning
    Management
    .NET
    .NET Core
    Neural Network
    OWIN
    Programming
    Programming Tools
    Recognition
    Security
    SQL Server
    UWP
    Visual Studio
    Web API
    Web Developer

    RSS Feed

    Latest Articles

© 2014-2020 ArthurWiz.com All Rights reserved. | Home | About |
Protected by Copyscape