Tuesday 25 September 2018

Introduction to Node Package Manager (npm)

What is npm?

  • package manager
  • task runner that can serve as a replacement for Gulp

How to install npm?


Upon installation user variable Path (in environment variables) gets a new entry:
C:\Users\user\AppData\Roaming\npm


How to see command line arguments?




How to check its version?




How to update npm?




Try the latest stable version of npm | npm Documentation

To verify it:



On Linux:

npm -v
6.14.4

npm install -g npm@latest
/home/bojan/.nvm/versions/node/v14.1.0/bin/npm -> /home/bojan/.nvm/versions/node/v14.1.0/lib/node_modules/npm/bin/npm-cli.js
/home/bojan/.nvm/versions/node/v14.1.0/bin/npx -> /home/bojan/.nvm/versions/node/v14.1.0/lib/node_modules/npm/bin/npx-cli.js
+ npm@6.14.5
updated 5 packages in 9.234s

npm -v
6.14.5


Packages


See here a complete definition of the package.

Node applications usually use (or depend on) multiple packages. There are three types of packages and each of them is listed within the object with the same name in packages.json:

  • regular (dependencies) - used in development in production
  • development (devDependencies) - packages are used only during application development and testing; we don't want to include them in the production and make users of our app unnecessarily download and build them 
  • optional (optionalDependencies) - dependencies which are used optionally - if they are found; their absence does not make application to fail

Configuration


npm gets its config settings from:

  • command line
  • environment variables
  • npmrc files
  • package.json file (in some cases)

.npmrc 


Global (per machine) 

Location: C:\Users\user\AppData\Roaming\npm\etc\npmrc

To find its location use:


>npm config get globalconfig
C:\Users\user\AppData\Roaming\npm\etc\npmrc

User-specific

Location:  %USERPROFILE%\.npmrc

To find its location use:

>npm config get userconfig
C:\Users\user\.npmrc

Local (per project) 

Location: in project's root directory.

It defines where can npm look for and fetch packages - package registries by listing their URLs:

# use npmjs registry by default
registry=https://registry.npmjs.org/

# use xyz registry for packages in @xyz scope
@xyz:registry=https://xyz.com/npm

After we change registry URL in .npmrc we need to do the following to force setting resolved field for all packages in packages-lock.json to this new domain:

  • delete node_modules
  • delete package-lock.json
  • run npm cache clean -f
  • run npm install



package.json


It lets npm know what the name of your package is as well as what dependencies it uses.

It is created and initialized via npm init command. This can be done retroactively - npm init can be executed for the already existing project...it will only add to it package.json file.

It is a manifest of your project that includes the packages and applications it depends on, information about its unique source control, and specific metadata like the project's name, description, and author. [source]

The biggest reason for using package.json to specify a project’s dependencies is portability. For example, when you clone someone else’s code, all you have to do is run npm i in the project root and npm will resolve and fetch all of the necessary packages for you to run the app. [source]

All modules from package.json are installed to ./node_modules/. [source] This is npm install's default behavior. [source]

npm opens and reads package.json from the current directory. If it can't find it, it issues an error like:


Working with package.json
package.json - Specifics of npm's package.json handling
Semantic Versioning
npm semver calculator
What's the difference between tilde(~) and caret(^) in package.json?


Properties


scripts


If within the project we have some tool we want to call frequently e.g. after every code change we don't want to type it every time but want to automate the process by adding a command within scripts object. See Babel example.



-w instructs npm to watch for changes in the src folder. Every time you make a change to a file in src, this command is automatically executed.

browser

Specifies alternative files to load in case bundling is done for the browser.

It is provided by a module author as a hint to javascript bundlers or component tools when packaging modules for client side use. [source]

Example:

"browser": {
 "vue": "vue/dist/vue.min.js"
},

package-lock.json

package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json. [package-lock.json - A manifestation of the manifest]

Is there a way to force npm to generate package-lock.json?
Shall package-lock.json be added to version control?


CLI commands

adduser
alias: login


>npm login
Username: bojan
Password:
Email: (this IS public) bojan@example.com

Logged in as bojan on https://registry.npmjs.org/.

If logging for the first time and if %USERPROFILE%/.npmrc does not already exist, it will be created with the content similar to this one:

//registry.npmjs.org/:_authToken=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx


cache
Used to add, list, or clean the npm cache folder.[npm-cache]
Try clearing the npm cache

cache clean

Cleans the npm cache.  Deletes all data out of the cache folder.

config
used to update and edit the contents of the user and global npmrc files.
Sub-commands:
config get

ci
npm ci (named after Continuous Integration) installs dependencies directly from package-lock.json and uses package.json only to validate that there are no mismatched versions. If any dependencies are missing or have incompatible versions, it will throw an error.  
Use npm ci if you need a deterministic, repeatable build. For example during continuous integration, automated jobs, etc. and when installing dependencies for the first time, instead of npm install.  [continuous integration - What is the difference between "npm install" and "npm ci"? - Stack Overflow]

npm ci don't use the package.json to install modules it uses the package-lock.json file. This ensures reproducible builds—you are getting exactly what you expect on every install.  [NPM CI is better than NPM install in your CI/CD - DEV



init
Creates package.json. It is used when creating/initializing a package (not when installing it).

install
Installs in the local node_modules folder all dependencies listed in package.json.

npm install reads package.json to create a list of dependencies and uses package-lock.json to inform which versions of these dependencies to install. If a dependency is not in package-lock.json it will be added by npm install.  
Use npm install to add new dependencies, and to update dependencies on a project. Usually, you would use it during development after pulling changes that update the list of dependencies but it may be a good idea to use npm ci in this case.  [source]

node.js - Why does "npm install" rewrite package-lock.json? - Stack Overflow 


install --production
npm will not install modules listed in devDependencies. Same applies when --production is omitted but the NODE_ENV environment variable is set to production.

install -g npm
Updates npm package itself.

install --save-dev package1 package2...
Installs packages locally (in  project's devDependencies)

To use npm registry at specific url (node.js - How to specify registry while doing npm install with git remote url? - Stack Overflow):
install --registry=registry_url

node.js - How to force NPM to use a locally hosted registry - Stack Overflow

list
Lists all packages installed locally (for the current project) and also their dependencies.

list --depth=0
Lists all packages installed locally but without their dependencies.

Example:



To check the version of locally installed package we can use:

$ npm list  | grep axis*
├─┬ axios@0.19.2

or

$ npm list --depth=0  | grep axis*
├── axios@0.19.2

list -g
Lists all packages installed globally and also full path to the installation directory.
On Windows, that directory is: C:\Users\user\AppData\Roaming\npm.

Example:



list -g --depth=0
Lists globally installed packages but not their dependencies.

Example:


login
see adduser

run script_name
run-script script_name
Runs an arbitrary command (script) from a package's scripts key (scripts object in package.json). Optional argument: command. If no command is provided, it will list the available scripts (all properties of "scripts" object). [npm-run-script]

npm run allows you to pass arguments to the script you run by putting them after a double dash:
npm run script_name -- --arg1 --arg2


outdated
Lists installed packages with their currently installed versions, highest versions they can be upgraded to (this info is deducted from semver info for each package in package.json) and the latest version currently available in public package repository.

$ npm outdated
Package            Current    Wanted   Latest  Location
@types/flat         0.0.28    0.0.28    5.0.0  my-node-project
@types/jest        24.0.21    24.9.1   25.2.1  my-node-project
@types/node        11.15.2  11.15.12  13.13.4  my-node-project
axios               0.18.1    0.18.1   0.19.2  my-node-project
dotenv               7.0.0     7.0.0    8.2.0  my-node-project
flat                 4.1.0     4.1.0    5.0.0  my-node-project
https-proxy-agent    3.0.1     3.0.1    5.0.0  my-node-project
jest                24.9.0    24.9.0   25.5.3  my-node-project
pg                  7.12.1    7.18.2    8.0.3  my-node-project
ts-jest             24.1.0    24.3.0   25.4.0  my-node-project
tslib               1.10.0    1.11.1   1.11.1  my-node-project
tslint              5.20.0    5.20.1    6.1.2  my-node-project
typescript           3.6.4     3.8.3    3.8.3  my-node-project


Wanted values are deducted from semver info in package.json:

  "devDependencies": {
    "@types/jest": "^24.0.15",
    "@types/node": "^11.13.17",
    "jest": "^24.8.0",
    "ts-jest": "^24.0.2",
    "tslib": "^1.10.0",
    "tslint": "^5.18.0",
    "typescript": "^3.5.3"
  },
  "dependencies": {
    "@types/flat": "0.0.28",
    "axios": "^0.18.1",
    "dotenv": "^7.0.0",
    "flat": "^4.1.0",
    "https-proxy-agent": "^3.0.1",
    "pg": "^7.11.0"
  }

Let's analyze this on the example of axios package:

npm list | grep axios
├─┬ axios@0.18.1

$ npm info axios

axios@0.19.2 | MIT | deps: 1 | versions: 42
Promise based HTTP client for the browser and node.js
https://github.com/axios/axios

keywords: xhr, http, ajax, promise, node

dist
.tarball: https://artifactory.srv.int.avast.com/artifactory/api/npm/npm/axios/-/axios-0.19.2.tgz
.shasum: 3ea36c5d8818d0d5f8a8a97a6d36b86cdc00cb27

dependencies:
follow-redirects: 1.5.10 

maintainers:
- mzabriskie <mzabriskie@gmail.com>
- nickuraltsev <nick.uraltsev@gmail.com>

dist-tags:
latest: 0.19.2       next: 0.19.0-beta.1  

published 3 months ago by emilyemorehouse <emilyemorehouse@gmail.com>

$ npm update axios

The last command does not do anything as we have:

"axios": "^0.18.1"

and as 0.x.x is special case, this means >=0.18.1 but < 0.19.0 (look here the rule: Semver cheatsheet).

So to update axios to the latest available version we can do:

$ npm uninstall axios
$ npm install axios

...after which in package.json we have:

"axios": "^0.19.2"


search search_terms...
Searches the registry for packages matching the search terms. [npm-search]

start
(short for run start)
npm runs the start script which is a command defined under scripts key in packages.json. This command usually starts the application with special configuration options (all listed in packages.json).

uninstall
uninstalls a package, completely removing everything npm installed on its behalf.
-S, --save: Package will be removed from your dependencies.
-D, --save-dev: Package will be removed from your devDependencies.
-O, --save-optional: Package will be removed from your optionalDependencies.

If all packages are uninstalled package.json contains empty dependency list:


"dependencies": {}



uninstall -g package_name
uninstall --global package_name
Uninstalls globally installed package

update
Updates local packages to their latest versions allowed by the version specified with syntax ^ ("compatible with") or ~ ("reasonably close to") (see here for explanation in detail)  in package.json. Installs missing packages. Changes package.json and package-lock.json.
npm install vs. update - what's the difference?
How do I update devDependencies in NPM?


update -g 
Updates global packages.

view package_name
Shows data about a package. This can be used to check if some package actually exists as if package doesn't exist a message "404 Not found : package_name" is printed out. [npm-view]

Example:

$ npm view axios

axios@0.19.2 | MIT | deps: 1 | versions: 42
Promise based HTTP client for the browser and node.js
https://github.com/axios/axios

keywords: xhr, http, ajax, promise, node

dist
.tarball: https://registry.npmjs.org/axios/-/axios-0.19.2.tgz
.shasum: 3ea36c5d8818d0d5f8a8a97a6d36b86cdc00cb27
.integrity: sha512-fjgm5MvRHLhx+osE2xoekY70AhARk3a6hkN+3Io1jc00jtquGvxYlKlsFUhmUET0V5te6CcZI7lcv2Ym61mjHA==
.unpackedSize: 346.1 kB

dependencies:
follow-redirects: 1.5.10 

maintainers:
- mzabriskie <mzabriskie@gmail.com>
- nickuraltsev <nick.uraltsev@gmail.com>

dist-tags:
latest: 0.19.2       next: 0.19.0-beta.1  

published 3 months ago by emilyemorehouse <emilyemorehouse@gmail.com>


    Packages


    Products (tools, apps) used to come with a single package which contained both the core functionality and CLI.  Nowadays products are moving away from coupling the CLI and library together. The CLI now usually lives in the xxx-cli package.

    Installing package locally vs globally


    Sometimes it is better to install a package locally (project by project) then globally. There are couple of reasons for this:
    • Different projects on the same machine can depend on different versions of the package allowing you to update them individually.
    • Not having an implicit dependency on the environment you are working in makes your project far more portable and easier to setup.

    Where to install development dependencies?


    Shall development dependencies (e.g. Rollup, Webpack, Browserify, Gulp...) be installed globally or locally? Looking at reasons listed in the previous paragraph it makes more sense to install them locally. Benefits:
    • if required, different versions of these tools can be used in different projects 
    • all required dependencies can be installed in one go with npm install. There is no need for extra steps for installing packages globally (which, in turn, might also require root or admin privileges opening potential security holes...)

    How to run locally installed packages?


    • use npx to run their binaries directly (usually for some quick demonstrations, not in the real-world projects)
    • add them to scripts section of package.json and call them via npm run
    • add them to gulpfile and then Gulp takes care of executing them

    How to find quickly the version of some package installed on the local machine?


    >npm list dtrace-provider
    my-app@0.1.0 C:\dev\my-app
    `-- web-ext@2.9.1
    `-- bunyan@1.8.12
    `-- dtrace-provider@0.8.7

    Use -g to look for package installed globally on the machine.

    Find the version of an installed npm package

    How to move installed package from production to dev?

    Move a module from devDependencies to dependencies in npm package.json

    npm install <module_name> --save-prod
    npm install <module_name> --save-dev


    Test frameworks should be installed as dev dependencies.
    E.g. from Getting Started · Jest:

    npm install --save-dev jest

    TypeScript should (in standalone apps, not packages intended for distribution) be installed as dev dependencies.
    E.g. From DefinitelyTyped/DefinitelyTyped: The repository for high quality TypeScript type definitions.?:

    npm install --save-dev @types/node


    How to update all locally installed packages?


    From Updating packages downloaded from the registry | npm Documentation:

    We recommend regularly updating the local packages your project depends on to improve your code as improvements to its dependencies are made.

    From How to update all the Node dependencies to their latest version:

    Some of those updates are major releases. Running npm update won’t update the version of those. Major releases are never updated in this way because they (by definition) introduce breaking changes, and npm want to save you trouble.
    To update to a new major version all the packages, install the npm-check-updates package globally (...).


    Instead of upgrading all of them in bulk (which can introduce breaking changes if major version is increased in update) I'd play safely and update packages one by one.


    Resources

    https://stackoverflow.com/questions/18412129/how-can-i-update-npm-on-windows
    https://www.npmjs.com/package/npm-windows-upgrade
    http://thecodebarbarian.com/3-neat-tricks-with-npm-run
    Common npm mistakes

    No comments: