We optimized our Pub/Sub automation script, reducing execution time from over 1 hour to less than 30 seconds using Node.js. Here’s a quick overview of the improvements:

1. Initial Python Script: Sequential processing, took over 1 hour.

2. Improved Python Script: Asynchronous processing, reduced time to 5 minutes.

3. Node.js Script: Further optimized, reduced time to less than 30 seconds.

From Over 1 Hour to Less Than 30 Seconds: Optimizing Pub/Sub Automation

In our quest to enhance performance, we transformed our Pub/Sub automation script from a slow, sequential process to a lightning-fast, asynchronous one. Here’s how we did it.

Initial Python Script

Our initial script was simple but slow, taking over 1 hour to create Pub/Sub topics and subscriptions sequentially.

from google.cloud import pubsub_v1
import mysql.connector

def create_topic(project_id, topic_name):
    publisher = pubsub_v1.PublisherClient()
    publisher.create_topic(request={"name": publisher.topic_path(project_id, topic_name)})

def create_subscription(project_id, topic_name, subscription_name, filter_expression):
    subscriber = pubsub_v1.SubscriberClient()
    subscriber.create_subscription(request={
        "name": subscriber.subscription_path(project_id, subscription_name),
        "topic": subscriber.topic_path(project_id, topic_name),
        "filter": filter_expression,
    })

def fetch_data():
    conn = mysql.connector.connect(host="", port="3306", user="", password="", database="ecms")
    cursor = conn.cursor()
    cursor.execute("SELECT name FROM your_table")
    results = cursor.fetchall()
    conn.close()
    return results

def create_pubsub_resources(project_id, name):
    topic_name = f"topic-{name}"
    subscription_name = f"subscription-{name}"
    filter_expression = f'attributes.name = "{name}"'
    create_topic(project_id, topic_name)
    create_subscription(project_id, topic_name, subscription_name, filter_expression)

if __name__ == "__main__":
    project_id = "your-project-id"
    data = fetch_data()
    for (name,) in data:
        create_pubsub_resources(project_id, name)

Key Details:

• Sequential Processing: Each Pub/Sub resource is created one after the other.

• Execution Time: Over 1 hour due to the sequential nature and blocking I/O operations.

Improved Python Script

By introducing asynchronous processing, we reduced the execution time to 5 minutes.

import asyncio
from google.cloud import pubsub_v1
import mysql.connector

async def create_topic(publisher, project_id, topic_name):
    await asyncio.to_thread(publisher.create_topic, request={"name": publisher.topic_path(project_id, topic_name)})

async def create_subscription(subscriber, project_id, topic_name, subscription_name, filter_expression):
    await asyncio.to_thread(subscriber.create_subscription, request={
        "name": subscriber.subscription_path(project_id, subscription_name),
        "topic": subscriber.topic_path(project_id, topic_name),
        "filter": filter_expression,
    })

def fetch_data():
    conn = mysql.connector.connect(host="", port="3306", user="", password="", database="ecms")
    cursor = conn.cursor()
    cursor.execute("SELECT name FROM your_table")
    results = cursor.fetchall()
    conn.close()
    return results

async def create_pubsub_resources(project_id, name):
    publisher = pubsub_v1.PublisherClient()
    subscriber = pubsub_v1.SubscriberClient()
    topic_name = f"topic-{name}"
    subscription_name = f"subscription-{name}"
    filter_expression = f'attributes.name = "{name}"'
    await create_topic(publisher, project_id, topic_name)
    await create_subscription(subscriber, project_id, topic_name, subscription_name, filter_expression)

async def main():
    project_id = "your-project-id"
    data = fetch_data()
    tasks = [create_pubsub_resources(project_id, name) for (name,) in data]
    await asyncio.gather(*tasks)

if __name__ == "__main__":
    asyncio.run(main())

Key Details:

• Asynchronous Processing: Uses asyncio to run tasks concurrently.

• Execution Time: Reduced to 5 minutes by overlapping I/O operations.

Node.js Script

Finally, rewriting the script in Node.js reduced the execution time to less than 30 seconds.

const {PubSub} = require('@google-cloud/pubsub');
const mysql = require('mysql2');
const {promisify} = require('util');

async function createTopic(pubSubClient, topicName) {
    const topic = pubSubClient.topic(topicName);
    await topic.create();
}

async function createSubscription(pubSubClient, topicName, subscriptionName, filterExpression) {
    const topic = pubSubClient.topic(topicName);
    const subscription = topic.subscription(subscriptionName);
    await subscription.create({filter: filterExpression});
}

async function fetchData() {
    const connection = mysql.createConnection({host: "", port: 3306, user: "", password: "", database: "ecms"});
    const query = "SELECT name FROM your_table";
    const rows = await promisify(connection.query).bind(connection)(query);
    connection.end();
    return rows;
}

async function createPubSubResources(pubSubClient, name) {
    const topicName = `topic-${name}`;
    const subscriptionName = `subscription-${name}`;
    const filterExpression = `attributes.name = "${name}"`;
    await createTopic(pubSubClient, topicName);
    await createSubscription(pubSubClient, topicName, subscriptionName, filterExpression);
}

async function main() {
    const projectId = "your-project-id";
    const pubSubClient = new PubSub({projectId});
    const data = await fetchData();
    const tasks = data.map(({name}) =>
        createPubSubResources(pubSubClient, name)
    );
    await Promise.all(tasks);
}

main().catch(console.error);

Key Details:

• Non-Blocking I/O: Node.js handles I/O operations asynchronously, making it highly efficient for this task.

• Execution Time: Reduced to less than 30 seconds due to the efficient handling of concurrent operations.

Conclusion

By leveraging asynchronous programming and Node.js, we achieved a significant performance boost, reducing the execution time from over 1 hour to less than 30 seconds. This journey underscores the importance of optimizing code for better efficiency and scalability.

Node.js serves as a powerful platform for building server-side applications, but encountering errors, especially during local development, can disrupt progress. One such error is connect ECONNREFUSED ::1:80, commonly faced while attempting to connect to a local server.

Understanding the Error

The error message connect ECONNREFUSED ::1:80 signifies that the Node.js application is attempting to connect to the IPv6 loopback address (::1) on port 80, commonly used for HTTP traffic. However, the connection is being refused, leading to the error.

const axios = require("axios")

try {
    const { data } = await axios.get('http://localhost')
} catch (error) {
    console.log(error.message)
    // output: "connect ECONNREFUSED ::1:80"
}

Solution 1

You can fix the connect ECONNREFUSED ::1:80 error by explicitly specifying the IPv4 loopback address (127.0.0.1) instead of relying on the IPv6 (::1) address.

const axios = require("axios")

try {
    const { data } = await axios.get('http://127.0.0.1')
    // Success Response
} catch (error) {
    console.log(error.message)
}

Solution 2

Starting from Node.js 17 IPv6 connections are prioritized when making network calls. This is great for future-proofing your code, but it can cause problems if the server you're trying to reach only supports IPv4. This is where setDefaultResultOrder() comes in like a knight in shining armor.

const axios = require("axios")
const { setDefaultResultOrder } = require("dns");

setDefaultResultOrder("ipv4first");

try {
    const { data } = await axios.get('http://localhost')
    // Success Response
} catch (error) {
    console.log(error.message)
}

Solution 3

Degrading the version of Nodejs runtime to Nodejs 16 or lower can also fix the issue but it is not recommended.

Try Yourself

Conclusion

Resolving the connect ECONNREFUSED ::1:80 error involves understanding the intricacies of local network configurations and resolving conflicts that prevent the successful connection of your Node.js application. By implementing the suggested troubleshooting steps and potential fixes, developers can overcome this error, ensuring uninterrupted progress in their development environment.

We hope this guide helps you tackle the connect ECONNREFUSED ::1:80 error, enabling you to focus on building your Node.js applications without connectivity hurdles during local development.

Stay tuned for more insightful troubleshooting tips and solutions!

Working with currency symbols can be tricky in any programming language, but especially so in Node.js. Fortunately, the currency-symbol-map package provides an easy-to-use solution for handling currency symbols in your Node.js application. Whether you're building a financial application or simply need to display currency symbols in your UI, currency-symbol-map can simplify the process for you.

Installation:

To use currency-symbol-map in your Node.js application, you first need to install it via npm. Open your terminal or command prompt and run the following command:

npm install currency-symbol-map

Usage:

Once you've installed currency-symbol-map, you can use it in your code like this:

const currencySymbolMap = require('currency-symbol-map');

console.log(currencySymbolMap('USD')); // outputs '$'
console.log(currencySymbolMap('EUR')); // outputs '€'
console.log(currencySymbolMap('JPY')); // outputs '¥'

As you can see, using currency-symbol-map is as simple as requiring the package and passing in a currency code as a parameter.

Examples:

Let's take a look at some practical examples of using currency-symbol-map in real-world scenarios.

Example 1: Displaying Currency Symbols in a UI

Suppose you're building an e-commerce website and need to display the currency symbol next to the product price. You can easily accomplish this with currency-symbol-map:

const currencySymbolMap = require('currency-symbol-map');

const productPrice = 10.99;
const currencyCode = 'USD';

const currencySymbol = currencySymbolMap(currencyCode);

console.log(`${currencySymbol}${productPrice}`); 
// outputs '$10.99'

Example 2: Formatting Currency Values

In this example, we have an array of products with different currencies. We use the forEach method to loop through each product and get the currency symbol using the currencySymbolMap package. We then format the price by concatenating the currency symbol with the price rounded to 2 decimal places using the toFixed method. Finally, we log the name of the product and the formatted price to the console.

const currencySymbolMap = require('currency-symbol-map');

const products = [
  { name: 'Product A', price: 10.99, currencyCode: 'USD' },
  { name: 'Product B', price: 9.99, currencyCode: 'EUR' },
  { name: 'Product C', price: 15.99, currencyCode: 'JPY' }
];

products.forEach(product => {
  const { name, price, currencyCode } = product;
  const currencySymbol = currencySymbolMap(currencyCode);
  const formattedPrice = `${currencySymbol}${price.toFixed(2)}`;
  console.log(`${name}: ${formattedPrice}`);
});

// Output:
// Product A: $10.99
// Product B: €9.99
// Product C: ¥15.99

Conclusion:

The currency-symbol-map package provides a simple and effective way to work with currency symbols in your Node.js application. Whether you're working with financial data or simply need to display currency symbols in your UI, currency-symbol-map can help you streamline your development process.

Image by rawpixel.com on Freepik

Axios is a popular JavaScript library used for making HTTP requests from a web page or Node.js application. When developing APIs, it can be helpful to convert an Axios request to a cURL command for debugging or testing purposes. Fortunately, the axios-curlirize npm package makes it easy to do just that. In this tutorial, we'll show you how to use axios-curlirize to convert Axios requests to cURL commands.

Prerequisites

Before we get started, make sure you have the following installed:

• Node.js (version 12 or higher)

• NPM (Node Package Manager)

Installing axios-curlirize

To use axios-curlirize, you need to install it using NPM. Open your terminal and run the following command:

npm install axios-curlirize

Using axios-curlirize

Once you have axios-curlirize installed, you can start using it to convert Axios requests to cURL commands. Here's an example:

const axios = require('axios');
const axiosCurlirize = require('axios-curlirize');

axiosCurlirize(axios);

axios.get('https://jsonplaceholder.typicode.com/posts/1')
  .then(response => {
    console.log(response.data);
  })
  .catch(error => {
    console.error(error);
  });

In this example, we import axios and axios-curlirize. We then call axiosCurlirize with the axios instance as an argument to enable cURL logging for Axios requests. After that, we make a GET request to https://jsonplaceholder.typicode.com/posts/1 using Axios.

When you run this code, you'll see the cURL command for the Axios request logged to the console, along with the response data:

$ node app.js
$ curl 'https://jsonplaceholder.typicode.com/posts/1' -H 'User-Agent: axios/0.21.1' -H 'Accept: application/json, text/plain, */*' -H 'Host: jsonplaceholder.typicode.com' -H 'Connection: keep-alive'
{ userId: 1,
  id: 1,
  title: 'sunt aut facere repellat provident occaecati excepturi optio reprehenderit',
  body: 'quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto'
}

And that's it! You can now use axios-curlirize to convert your Axios requests to cURL commands for debugging or testing purposes. It's a simple and effective tool to have in your developer toolkit.