Tag / NodeJs

by Aaron

How to list all dependents of a Node package

You can use the web npm web site as it lists the dependents for any given package.

Like coffee-script for example:

http://npmjs.org/browse/depended/coffee-script

Or, you could get JSON results for the same query (using curl) (just substitute the name of the package with "sql" in the example: 

> curl -g 'http://registry.npmjs.org/-/_view/dependedUpon?group_level=2&startkey=["sql"]&endkey=["sql","ZZZZZZZZZZZZZ"]&skip=0&limit=1000'

Results:

{"rows":[
{"key":["sql","anydb-sql"],"value":1},
{"key":["sql","dbal"],"value":1},
{"key":["sql","ectypes-downstairs"],"value":1},
{"key":["sql","fixr"],"value":1},
{"key":["sql","fixr-compiled"],"value":1},
{"key":["sql","forerunner-postgres-store"],"value":1}
{"key":["sql","pg-dal"],"value":1},
{"key":["sql","relational"],"value":1},
{"key":["sql","sequelize"],"value":1},
{"key":["sql","sql-generate"],"value":1},
{"key":["sql","sqlbox"],"value":1},
{"key":["sql","triplie"],"value":1},
{"key":["sql","voltron-postgres-adapter"],"value":1},
{"key":["sql","worm"],"value":1}
]}

The second value in the key array is the name of the package dependent on the sql package.

This of course only will find other published packages, and doesn’t help find applications, etc. that are using a specific package.

by Aaron

Loading Models in NodeJS

I’d answered a question on StackOverflow about where to put “models” in a NodeJS project. I wanted to elaborate on the simple auto loader I use to load a folder full of models (and I use this pattern other places as well). Normally, I create a folder called models:

image

Inside the Models folder, I have a file called, models.js.

Inside another module (like app.js), I’ve got a line of code that looks like this: 

var models = require('./models').initialize(app, services);

While the require (reference) by default looks for a file called index.js in the folder models, I’ve in this case added a simple package.json file with an override: 

{
    "main": "./models.js"
}

The reason I do this is that in a tabbed source code editor, having several index.js files can be confusing to see. So, this allows me to name the file something that is more memorable and understandable at a glance.

Inside of the models.js class, I’ve written code to automatically load all of the models (and call an initialize function once for each module): 

var initialize = function(app, services) {
    var models = {};
    var currentFile = path.basename(__filename); // just file name
    var modelFiles = fs.readdirSync(__dirname);
    // loop through all of the files in the current directory
    for(var i= 0, len = modelFiles.length; i < len; i++) {
        // ignore this file (via global NodeJS variable)
        if(modelFiles[i] === currentFile ||
            path.extname(modelFiles[i]).toLowerCase() !== '.js') {
            continue;  // skip the current file and anything without a "JS" extension
        }
        // require it
        var model = require(path.join(__dirname, modelFiles[i]));
        // call its intialize
        model.initialize(services, models);
    }
    exports.models = models;
    return models;
};

exports.initialize = initialize;

The logic is simple enough – grab all of the files in the current path, then loop through each, filtering the current file and any that don’t end with “js”. This logic could be adjusted of course to reflect other coding styles and requirements.

Finally, each module is loaded and then an initialize method is called (admittedly, I could make it more robust by checking for the existence of the initialize function before calling it (but in this case, I know each module should be initialized in a particular way).

by Aaron

Sending a socket to a forked process in Node.JS

If you want to fork a process in Node and pass a socket, the current Node documentation has a rather odd example, so I’ve simplified it here for my own sanity (further complicated by the fact that the WebStorm debugger can’t debug a forked Node.JS process, which confused me for too long). Hopefully someone else finds this useful at some point.

Step 1, in a file called app.js:

var child = require('child_process').fork('socket_handler.js');

// Open up the server and send sockets to child
var server = require('net').createServer();

server.on('connection', function (socket) {
    child.send('socket', socket);
    server.getConnections(function(err, count) {
        console.log("Connections: " + count);
    });
});
server.listen(1337);

And then, in a file called socket_handler.js which is located in the same directory as app.js:
process.on('message', function(message, socket) {
    socket.on('data', function(data) {
        // really poor echo ... :)
        socket.write(data);
    });
});

In this case, the fork happens upon application startup by calling the fork method of the child_process module. It starts by executing the code in socket_handler.js. As this is a new instance of the V8 engine used by Node.JS, remember there’s a sizable overhead to a forked process.

After creating the server and listening on port 1337, when a connection is made (which can be tested by a terminal emulator or telnet), the socket is passed to the forked process. In fact, it’s not really the object as much as it’s the handle to the new socket.

The send method takes two parameters, a string message which can be any identifier you want, and the the handle, or socket in this case.

The forked process receives the message via the message event on the process. If you’re sending more than one message, you can add conditional logic to handle the type of message in the event handler. As part of the call to fork, you can also pass command line arguments and read them using process.argv.

by Aaron

Simple Node based Http Put simulator

It’s great what you can accomplish in a few lines of code. The Node based source code below uses express to create a mini view-based web server along with a mock Http Put file upload destination. 


/**
 * Module dependencies.
 */

var express = require('express')
  , routes = require('./routes')
  , http = require('http')
  , path = require('path');

var app = express();

app.configure(function(){
  app.set('port', process.env.PORT || 8080);
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(path.join(__dirname, 'public')));
});

app.put('/upload/:fileUpload', function(req, res, next){
    console.log('uploading!');
    res.send('OK', { 'Content-Type': 'text/plain' }, 200);
});

app.configure('development', function(){
  app.use(express.errorHandler());
});

app.get('/', routes.index);

http.createServer(app).listen(app.get('port'), function(){
  console.log("Express server listening on port " + app.get('port'));
});

Using WebStorm’s Node/Express template, I only added 4 lines: 

app.put('/upload/:fileUpload', function(req, res, next){
    console.log('uploading!');    
    res.send('OK', { 'Content-Type': 'text/plain' }, 200);
});

These 4 lines added a new route/path for uploading a file in the pattern of /upload/ {fileName}.

The response was “OK.”

I’d tried this same thing using ASP.NET MVC 4 (with Razor), but was stumped by the error when trying to use HttpPut on an Action in the controller.

I was using this end-point as a mock-upload destination for my SmugMug application, SnugUp. I wanted to eliminate the uploading process as it’s time consuming and messes up my SmugMug galleries! Smile

The C# code is simple as it uses HttpWebRequest:

HttpWebRequest uploadRequest = (HttpWebRequest)WebRequest.Create(ApplicationConstants.UploadUrl + "photo.jpg");

uploadRequest.Timeout = (int)10080 * 60 * 1000; // 7 days 
uploadRequest.Method = "PUT";
uploadRequest.UserAgent = ApplicationConstants.UserAgent;
uploadRequest.ContentLength = fi.Length;
uploadRequest.KeepAlive = true;

(SmugMug grabs the file name from a custom Http Header, so, the put URL is always “photo.jpg”.)