A YouTube Downloader App using ExpressJS, NodeJS and Custom Code - Kevin Uriel Fonseca
Background
Let’s start by saying this was just a fun project that I wanted to create in order to have something new as part of my portfolio. Since every single website that I know about that provide this type of service are still made with PHP and SQL, I decided to use JavaScript which nowadays is the to-go development tool that everyone seems to be using.
The functionalities are simple, find and fetch data from a given YouTube URL and displays it to the front-end user. What I needed for this project was three sections, a form with a text input; a left div containing the video that was being fetched and a right div that is going to work as the container for all the videos previously retrieved – so the user can go back when he/she is ready to download X video but… – by the user.
All of that is mainly for the front-end side of the application which I’ll talk about later on. Right now, let’s talk about the server which makes all of this possible!.
Setting up the Server
Assuming you’re working on a pre-made project, please install the npm ytdl-core library by using the following command:
npm i ytdl-coreFurthermore, we now have the need to make use of it and for us to make it happen, we will create several HTTP endpoints.
const express = require('express');const { getYouTubes, getYouTube, createYouTube, deleteAllYoutube} = require('../../../controllers/extras/youtube');const router = express.Router();/** https://yourwebsite.com/api/v1/extras/youtube*/router.route('/').get(getYouTubes);/** https://yourwebsite.com/api/v1/extras/youtube/getinfo* This one is a POST request meaning that it can receive data from the front-end*/router.route('/getinfo').post(createYouTube);/** https://yourwebsite.com/api/v1/extras/youtube/deleteall* This is the function that runs the 15 of each month with a cron-job configured*/router.route('/deleteall').delete(deleteAllYoutube);/** https://yourwebsite.com/api/v1/extras/youtube/0123/4567* When accessing a single video page, the data will be retrieved from this endpointt*/router.route('/:id/:videoid').get(getYouTube);module.exports = router;The code above contains some comments in which I tried to give a summary about what the purpose of each endpoint will be about. Obviously, this might be a bit confusing for beginners who are just getting into the NodeJS enviroment but believe me once you have created an endpoint, understanding the concept of REST APIs is waaaay toooooo easyyyyy.
How does our logic code looks like?
In computer science and IT in general when someone says the infrastructure and/or logic they are talking about the process that handles the requests or X function.
A good example is our first route; said route only needs to fetch the data from our database without any type of filtering and the perfect way to achieve that is by simply setting our YouTube model and calling its find() method; obviously if you want to apply some filtering then you will need to edit your find() into something like find({ x: y }) where X is the name and y is the value you want to find in X. Take a look into what it looks like:
// @desc Get all youtube// @route GET /api/v1/extras/youtube// @access Private// @status DONEexports.getYouTubes = asyncHandler(async (req, res, next) => { const youtube = await YouTube.find(); res.status(200).json({ success: true, data: youtube });});Secondly, as a developer you already know what you’re looking for if you’re reading this post. I’m assuming what you need to know is the function that makes the YouTube app work, right? Well, I’ll put it easy; that and because I do not want to make this post longer.
As mentioned before the ytdl-core is the library being used to retrieve data from the YouTube data.
// @desc Get youtube video data// @route GET /api/v1/extras/youtube/getinfo// @access Private// @status DONEexports.createYouTube = asyncHandler(async (req, res, next) => { /* * * req.body usually contains the data coming from the front-end, commonly via a form/input element. * It is also important to note that said data comes according to the name of the input * e.x: <input name="password" type="password" value="THIS_IS_MY_PASSWORD" /> * e.x2: <input name="title" type="text" value="THIS_IS_MY_TITLE" /> * * The way to retrieve them on express with req.body would look likes this * const { password, title } = req.body * or * req.body.password and req.body.title. Each of these lines will then save * the data( THIS_IS_MY_PASSWORD and THIS_IS_MY_TITLE) into a temporary memory. */ const videoUrl = req.body.video_url // If not videoUrl given, throw an error - self-explanatory if (!videoUrl) { return next(new ErrorResponse(`Please add a Url to shorten`, 400)) } // Check videoUrl is valid - not necessary but this just assures the given URL is properly formatted. if (!validUrl.isUri(videoUrl)) { return next(new ErrorResponse(`Invalid URL`, 400)) } // Validate YouTubeURL ytdl.validateURL(videoUrl) /* * * Once the URL validation has turned out to be "green", then we can proceed to the following function of getinfo() which as titled, * takes and gets the info regarding the video of given URL * */ // Get info from Video const video = await ytdl.getInfo(videoUrl) /* * * Once the video has been found, I now want to retrieve all the formats that are played as videos by using the function * filterFormats and store the data into memory within a variable called videoFormats; * */ // Find ONLY video NO AUDIO formats to choose from const videoFormats = ytdl.filterFormats(video.formats, 'videoonly') /* * * From all those video formats found, I now need the best quality one. * The chooseFormat() function which takes two parametes with the first one being the formats and * a second parameter which specifies the quality will make this possible. * */ // Find the highest ONLY video NO AUDIO to download const highQualityOnlyVideo = ytdl.chooseFormat(videoFormats, { quality: 'highest', }) /* * * * With that being said, if for some reason you want to generate links to download the audio * from a given video, we can again make use of the filterFormats() function and this time, look for audioonly * * */ // Find ONY audio NO VIDEO formats to choose from const audioFormats = ytdl.filterFormats(video.formats, 'audioonly') /* * * * Found the audio? thats great, now find the best quality audio available * * */ // Find the highest ONLY audio NO VIDEO to download const highQualityOnlyAudio = ytdl.chooseFormat(audioFormats, { quality: 'highest', }) /* * * * Lastly the final video with the best audio and visuals found. * * */ // Find the video to download const finalVideo = ytdl.chooseFormat(video.formats, { quality: 'highest', }) res.status(200).json({ success: true, data: video, })});This in fact might be great but it puts too many restrictions in our end; these include having to make several request to the same URL and/or having to download the files in one sitting even if you’re not ready yet. This is reason, I usually like to store data in my own db even if not really necessary. Making this is easy, we only need to set a create() method using our YouTube model:
// Create documentconst youtube = await YouTube.create({ videoId: video.videoDetails.videoId, title: video.videoDetails.title, text: video.videoDetails.description, videoFetchedUrl: video.videoDetails.video_url, videoEmbedUrl: video.videoDetails.embed?.iframeUrl, videoToDownload: { url: finalVideo.url, }, videoOnly: { url: highQualityOnlyVideo.url, }, audioOnly: { url: highQualityOnlyAudio.url, }, thumbnails: video.videoDetails.thumbnails.map((image) => image.url), author: { name: video.videoDetails.author?.name, link: video.videoDetails.author?.user_url, channelUrl: video.videoDetails.author?.channel_url, }, category: video.videoDetails.category, likes: video.videoDetails?.likes, dislikes: video.videoDetails?.dislikes, keywords: video.videoDetails.keywords ? video.videoDetails.keywords .map((keyword) => keyword.trim().replace(/ +/g, ' ').split(' ').join('-').toLowerCase() ) .filter((keyword) => keyword.length !== 0) : 'no-tags', views: video.videoDetails?.viewCount,})The Final Step
Let’s say that whenever we make a request to ytdl.getInfo(url) most of the info we need is found within an object called videoDetails which contains said info and way more data that might be good to your project. Now that’s the usual way to store data into a db and now instead of returning the video object, we will return the data that was saved in our db by typing youtube in the data variable that is located inside our json object:
res.status(200).json({ success: true, data: youtube})Now that’s pretty much everything you need to know. If you already know about how mongoose works with MongoDB, you can simply copy and paste the functions and models below into your project; you can also modify the endpoints to fit your own needs.
The To-Copy-Paste Code
const ytdl = require('ytdl-core');const validUrl = require('valid-url');const YouTube = require('../../models/extras/YouTube');const ErrorResponse = require('../../utils/ErrorResponse');const asyncHandler = require('../../utils/async');// @desc Get all youtube// @route GET /api/v1/extras/youtube// @access Private// @status DONEexports.getYouTubes = asyncHandler(async (req, res, next) => { const youtube = await YouTube.find(); res.status(200).json({ success: true, data: youtube });});// @desc Get single youtube// @route GET /api/v1/extras/youtube/:id/:videoId// @access Private// @status DONEexports.getYouTube = asyncHandler(async (req, res, next) => { const youtube = await YouTube.findOne({ _id: req.params.id, videoId: req.params.videoid }); if (!req.params.id.match(/^[0-9a-fA-F]{24}$/) || !youtube) { return next( new ErrorResponse(`Document not found with id of ${req.params.id}`, 404) ); } res.status(200).json({ success: true, data: youtube });});// @desc Get youtube video data// @route GET /api/v1/extras/youtube/getinfo// @access Private// @status DONEexports.createYouTube = asyncHandler(async (req, res, next) => { const videoUrl = req.body.video_url; // If not videoUrl given, throw an error if (!videoUrl) { return next(new ErrorResponse(`Please add a Url to shorten`, 400)); } // Check videoUrl is valid if (!validUrl.isUri(videoUrl)) { return next(new ErrorResponse(`Invalid URL`, 400)); } // Validate YouTubeURL ytdl.validateURL(videoUrl); // Get info from Video const video = await ytdl.getInfo(videoUrl); // Find ONLY video NO AUDIO formats to choose from const videoFormats = ytdl.filterFormats(video.formats, 'videoonly'); // Find the highest ONLY video NO AUDIO to download const highQualityOnlyVideo = ytdl.chooseFormat(videoFormats, { quality: 'highest' }); // Find ONY audio NO VIDEO formats to choose from const audioFormats = ytdl.filterFormats(video.formats, 'audioonly'); // Find the highest ONLY audio NO VIDEO to download const highQualityOnlyAudio = ytdl.chooseFormat(audioFormats, { quality: 'highest' }); // Find the video to download const finalVideo = ytdl.chooseFormat(video.formats, { quality: 'highest' }); // Create document and store it to MongoDB const youtube = await YouTube.create({ videoId: video.videoDetails.videoId, title: video.videoDetails.title, text: video.videoDetails.description, videoFetchedUrl: video.videoDetails.video_url, videoEmbedUrl: video.videoDetails.embed?.iframeUrl, videoToDownload: { url: finalVideo.url }, videoOnly: { url: highQualityOnlyVideo.url }, audioOnly: { url: highQualityOnlyAudio.url }, thumbnails: video.videoDetails.thumbnails.map((image) => image.url), author: { name: video.videoDetails.author?.name, link: video.videoDetails.author?.user_url, channelUrl: video.videoDetails.author?.channel_url }, category: video.videoDetails.category, likes: video.videoDetails?.likes, dislikes: video.videoDetails?.dislikes, keywords: video.videoDetails.keywords ? video.videoDetails.keywords .map((keyword) => keyword .trim() .replace(/ +/g, ' ') .split(' ') .join('-') .toLowerCase() ) .filter((keyword) => keyword.length !== 0) : 'no-tags', views: video.videoDetails?.viewCount }); res.status(200).json({ success: true, data: youtube });});// @desc Delete all youtube links// @route DELETE /api/v1/extras/youtube/deleteall// @access Private// @status DONEexports.deleteAllYoutube = asyncHandler(async (req, res, next) => { await YouTube.deleteMany(); res.status(200).json({ success: true, data: {} });});Model:
const mongoose = require('mongoose');const slugify = require('slugify');const YouTubeSchema = new mongoose.Schema( { videoId: { type: String, trim: true, required: [true, 'Please add a name'], default: '0' }, title: { type: String, trim: true, required: [true, 'Please add a name'], default: 'Unknown' }, slug: { type: String, required: false }, text: { type: String, trim: true, required: [true, 'Please add a name'], default: 'Unknown' }, videoFetchedUrl: { type: String, trim: true, required: [true, 'Please add videoFetched url'], default: '...' }, videoEmbedUrl: { type: String, trim: true, required: [true, 'Please add videoFetched url'], default: '...' }, videoToDownload: { url: { type: String, trim: true, required: [true, 'Please add videoToDownload url'], default: '...' } }, videoOnly: { url: { type: String, trim: true, required: [true, 'Please add videoOnly url'], default: '...' } }, audioOnly: { url: { type: String, trim: true, required: [true, 'Please add an audioOnly url'], default: '...' } }, thumbnails: { type: [String], trim: true, required: [true, 'Please add an audioOnly url'], default: '...' }, author: { name: { type: String, trim: true, required: [true, 'Please add author'], default: 'Unknown' }, link: { type: String, trim: true, required: [true, 'Please add author link'], default: 'Unknown' }, channelUrl: { type: String, trim: true, required: [true, 'Please add channel url'], default: 'Unknown' } }, category: { type: String, trim: true, required: [true, 'Please add category'], default: 'Uncategorized' }, likes: { type: Number, required: [false, 'Please display likes'], default: 0 }, dislikes: { type: Number, required: [false, 'Please add dislikes'], default: 0 }, keywords: { type: [String], required: [true, 'Please add a term id option'], default: 'no-tags' }, views: { type: Number, required: [false, 'Please add views'], default: 0 } }, { toJSON: { virtuals: true }, toObject: { virtuals: true }, timestamps: true, id: false });// Create slugYouTubeSchema.pre('save', function (next) { this.slug = slugify(this.title, { lower: true }); next();});module.exports = mongoose.model('YouTube', YouTubeSchema);On Another Note
If you would like to delete the data every certain hour, day, month, year, etc; you can simply create cron-job by installing node-cron:
npm i node-cronSetting it up is simple, call the library and schedule the cron-jon according to your needs in this way:
const cron = require('node-cron')const { deleteAllYoutube } = require('../controllers/extras/youtube')const cronJobs = async (req, res, next) => { /* * * ON THE 15 OF EACH MONTH * */ cron.schedule( '* * 15 * *', async () => { await deleteAllYoutube() } )}module.exports = cronJobsI would highly recommend to create a file and name it as cronJob.js. You can have your cron-jobs logic separated from your server file and instead simply call the function in the server file:
/* * * SERVER/INDEX.js file * *//* MORE CODE ABOVE*/const cronJobs = require('./YOUR_FOLDER/cronJobs')// Make all cron-jobs available in your server file.cronJobs()/*MORE CODE BELOW*/Don’t forget to check my previous articles on my blog and check this project on my portfolio age!.
Bye bye 🙂
If you want to know more about Web Development with NodeJS, take a look into this book:
