内容简介:The documentation in this repository describe the FullStack webscrapping platform for use in Machine learning.
The documentation in this repository describe the FullStack webscrapping platform for use in Machine learning.
Architecture
We first break the architecture into four distictive components namely Front-End, API, Scrapers and Database. The user sends information from the front-end to the API, the fron-end connects the API through a form. Inputs like the youtube URL are sent through front-end. Later the scrapers through the API pulls the necessary data and is saved to the database. Afterwhich the data is served to the front-end.
The Tech Stack are as below
- Front-End - javascript
- API - express
- scraper - puppeteer
- db - mysql (typeorm)
Also we need nodejs, npm and mysql.
The Architecture consists of several components:
Front End
For the Front-end we will have a header, an input box and a button. Below which we will have render boxes which renders relevant info from json. This will send data to the API.
API
We will have to create a single route with two methods GET and POST. We use nodejs and simple backed framework express.
Scraper
This function takes in URL and reaches out to YouTube, fetch the relevant data and then store it into the database.
Database
We use mySQL here. Here we add id, name, avatar and channelURL
To run the program
First go into server
$ npm install init
Install all the necessary packages
$ npm install express $ npm install body-parser
Run the index.js script
$ node index.js
Thanks to Aron from Uber
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
ActionScript 3.0 Cookbook中文版
Joey Lott、Adobe Dev Library / 陈建勋 / 电子工业 / 2007-11-01 / 78.00元
本书讨论了在Flash Player里执行的ActionScript3.0语言,采用问题—解法—讨论的形式讲解开发过程中常见问题的实际解法,例如:检测用户的 Flash Player 版本或操作系统;格式化日期和货币类型;接受用户输入及操作文字字符串;在运行时绘制各种形状;访问音频和视频;使用 Flash Remoting 进行远程过程调用;加载、发送和检索 XML 数据等。 全书涵盖客户端......一起来看看 《ActionScript 3.0 Cookbook中文版》 这本书的介绍吧!