LitArk » Books » Romance novel

Maignan - Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)

Here you can read online Maignan - Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1) full text of the book (entire story) in english for free. Download pdf and epub, get meaning, cover and reviews about this ebook. year: 2020, genre: Romance novel. Description of the work, (preface) as well as reviews are available. Best literature library LitArk.com created for fans of good reading and offers a wide selection of genres:

Romance novel Science fiction Adventure Detective Science History Home and family Prose Art Politics Computer Non-fiction Religion Business Children Humor

Choose a favorite category and find really read worthwhile books. Enjoy immersion in the world of imagination, feel the emotions of the characters or learn something new for yourself, make an fascinating discovery.

Book:
Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)
Author:
Maignan / Nicolas
Genre:
Books / Romance novel
Year:
2020
Rating:
5 / 5
Favourites:
Add to favourites
Your mark:
- 100
- 1
- 2
- 3
- 4
- 5

Description
Author's other books
Similar books

Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1): summary, description and annotation

We offer to read an annotation, description, summary or preface (depends on what the author of the book "Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)" wrote himself). If you haven't found the necessary information about the book — write in the comments, we will try to find it.

Maignan: author's other books

Who wrote Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)? Find out the surname, the name of the author of the book and a list of all author's works by series.

Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1) — read online for free the complete book (whole text) full work

Below is the text of the book, divided by pages. System saving the place of the last page read, allows you to conveniently read the book "Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)" online for free, without having to search again every time where you left off. Put a bookmark, and you can go to the page where you finished reading at any time.

Light

Font size:

↓

↑

Reset

Interval:

↓

↑

Bookmark:

Make

Python WebApp

Nicolas Maignan

The power of indecision

A code book taking you through a small Python Full Stack journey

0. Introduction

The goal of this book is to take you through the journey that I, author of this book, followed. This journey, despite being initially guided by my indecision, led to a drastic improvement in my programming skills and an increase in my value as a developer. The point of this book is for you to be able to replicate the process on your own use case, which you are free to define according to your own centers of interest or to your latest concerns. If you go through the following pages, you will learn how to create a Car Price Prediction Web-Application serving the results of a Machine-Learning model using the following frameworks:

Python:
- Environment control: Conda, Pip
- Quality control: Mypy, Flake8, Black
- Web crawling/scraping: Selenium
- General data science: Pandas, LightGBM, Scikit-learn
- Web-Applications: Quart (similar to Sanic, Flask, Django, etc.)
DevOps:
- App containerization: Docker
- Deployment: Docker-compose

In 2018, I was looking to buy a second hand car. My professional life had just started, my student loan was still to be paid back, and I had always heard that buying a brand-new car was the worst possible investment, as the vehicle loses 10 to 30% of its value from the first kilometer traveled. My quest for the best possible deal had already lasted several weeks, and where I started with a precise idea of the car I wanted to acquire, I was later standing completely crushed under my own indecision. Too many brands, too many models, to many prices, and way too many offers. I started to feel a lack of knowledge on my end. So many variables were to be considered that I was never able to asses whether a price was fair or not. The more lost I was, the more obsessed I became with the idea of making the best possible choice.

Luckily enough, I had already been introduced to the wonders of Data Science, Web Scraping, Machine Learning and DevOps. And it is faced with the idea of spending a new weekend of active but fruitless research that I decided to solve my problem in a slightly overkill way. In this book, I wont describe the exact development process I followed. Instead, Ill tell you how things should have been done in the first place to avoid some snags I encountered along the way. On this note, step zero should have been to clearly express the problem I was attempting to solve. From now on, this is the problematic were are trying to solve:

How to make sure the price of a second hand car is fair?

1. Working with data

1.0 Scoping the need

Answering this question in a structured way requires a lot of data. But after several weeks of intensive research on countless websites, the basic variables needed to describe a car are quite clear.

`brand` : the brand of this car

`model` : the model of this car

`mileage` : the distance already traveled by this car

`model_year` : the year in which this car was manufactured

`circulation_date` : the date at which this car was put to circulation

`transmission` : the kind of transmission of this car (automatic or manual)

`number_of_doors` : the number of doors of this vehicle

`din_horsepower` : the Deutsche Industrie Normen horsepower of this car

`tax_horsepower` : the taxable horsepower of this car

`energy` : the type of fuel or energy used by this car

`first_hand` : whether this car is a first-hand one

`imported` : whether this car was imported (to France)

`price` : and of course the exhibition price of this car

And despite a large number of variables set aside, we still end up with the need to retrieve thirteen data fields in a structured way, for several thousand announcements.

Of course such an open dataset, representing the current state of the second-hand car market, doesnt exist. It has to be created and to do so, were going to scrape a website on which those fields are structured enough that they can be easily retrieved. Finding the perfect website doesnt take long, although many websites still allow people to sell second-hand goods in a completely unstructured way, meaning that the data available from an ad to another are very inconsistent.

1.1 Acquiring data

Lets first define two terms, which are very close to one another:

Web scraping is the fact of automatically downloading web-pages while extracting data from them.
Web crawling is the fact of automatically downloading web-pages while extracting hyperlinks they contain and following those hyperlinks.

While web-scraping and crawling arent illegal by themselves, its highly probable that using those techniques on websites you dont own will be illegal. Sadly, the legislation concerning scraping and crawling the web is full of gray areas, and whether the websites Terms of Use are enforceable, whether you entered a contract with the website or whether fair use can be applied can only be determined case by case.

So before starting any data gathering based on crawling or scraping web-pages, seek legal advice, doubt anything you can read, and cover yourself. I dont think I will ever stop gathering data this way for very simple reasons:

I consider this data as being publicly available data.
Google and all Search Engine Providers have constantly been crawling the web since its start, indeed, web-pages and their content have to be discovered and indexed if results are to be displayed in a search engine (although Google gives webmasters the right to decide whether they want to be indexed on their search engine or not, and most of them want to).

But none of those reasons are legal arguments.

The least that can be done to cover yourself is to use a Virtual Private Network with a Kill-switch , but I would recommend using Proxy chains , picking proxies in several and politically opposed countries, even though this would slow the crawling/scraping process.

Whatever the nature of a project, developing sustainably way means controlling your environment. And by controlling, I dont mean applying all the possible best practices, but at least a set of rules to follow and which make your code more readable and maintainable over time. If your project becomes something greater than you initially forecasted, you, as well as the other possible contributors, will be grateful that this frame was set early on.

In this case, the programming language well use is Python3, and well use Anaconda to control our python environments. Anaconda is a free and open-source distribution of Python and R. Its build to simplify package version and environment management.

Lets build a research python environment which well be using for scraping, cleaning, and modelling:

conda create -n research python=3.7.5

conda activate research

Well need several python packages now, including:

Pandas
Numpy
Matplotlib
Seaborn
Jupyter
Selenium
LightGBM

pip install pandas numpy matplotlib seaborn jupyter selenium lightgbm

Now that this environment is ready, it is time to create a working directory and to save a static environment file, which allows us to build environments that consistently support our code. To do so, the following command can be run:

Light

Font size:

↓

↑

Reset

Interval:

↓

↑

Bookmark:

Make

Similar books «Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)»

Look at similar books to Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1). We have selected literature similar in name and meaning in the hope of providing readers with more options to find new, interesting, not yet read works.

Jonathan S. Walker

Machine Learning For Beginners

Kumar Alok

Practical Full Stack Machine Learning

Gnana Lakshmi T C

Hands-on Supervised Learning with Python: Learn How to Solve Machine Learning Problems with Supervised Learning Algorithms Using Python

Brady Ellison

Python Machine Learning. A Step by Step Beginner’s Guide to Learn Machine Learning Using Python

Haroon

Python Machine Learning Case Studies Five Case Studies for the Data Scientist

Karthik Ramasubramanian

Machine Learning Using R

Chris Albon

Machine learning with Python cookbook: practical solutions from preprocessing to deep learning

Ciaburro Giuseppe

Python Machine Learning Cookbook

Soranson

Python Machine Learning: How to learn Machine Learning with Python. The Complete Guide to Understand Python Machine Learning for Beginners and Artificial Intelligence

Trinity

Python Machine Learning: The Beginners Guide To Learn Python Machine Learning Including Keras, Numpy, Scikit Learn and PyTorch.

Matt Harrison

Machine Learning Pocket Reference: Working with Structured Data in Python

Mueller Andreas C.

Introduction to Machine Learning with Python

Reviews about «Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1)»

Discussion, reviews of the book Python WebApp: Learn how to serve a Machine Learning Model predicting car prices (Full stack Book 1) and just readers' own opinions. Leave your comments, write what you think about the work, its meaning or the main characters. Specify what exactly you liked and what you didn't like, and why you think so.