Difference between revisions of "Python for Data Science"

From Sinfronteras
Jump to: navigation, search
(→‎Keep a python script running on a remote server)
 
(273 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
https://www.amcharts.com/demos/multi-series-map/
 
  
 +
<br />
 +
For a standard Python tutorial go to [[Python]]
  
  
−
https://www.udemy.com/course/python-for-data-science-and-machine-learning-bootcamp/
+
<br />
 +
==Courses==
 +
*Udemy - Python for Data Science and Machine Learning Bootcamp
  
−
https://www.udemy.com/course/python-rest-apis-with-flask-docker-mongodb-and-aws-devops/
+
:https://www.udemy.com/course/python-for-data-science-and-machine-learning-bootcamp/
  
−
https://www.udemy.com/course/rest-api-flask-and-python/
 
  
−
https://www.udemy.com/course/react-django-full-stack/
+
<br />
 +
==Anaconda==
 +
Anaconda is a free and open source distribution of the Python and R programming languages for data science and machine learning related applications (large-scale data processing, predictive analytics, scientific computing), that aims to simplify package management and deployment. Package versions are managed by the package management system conda. https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)
 +
 
 +
En otras palabras, Anaconda puede ser visto como un paquete (a distribution) que incluye no solo Python (or R) but many libraries that are used in Data Science, as well as its own virtual environment system. It's an "all-in-one" install that is extremely popular in data science and Machine Learning.Creating sample array for the following examples:
 +
 
 +
 
 +
<br />
 +
===Installation===
 +
Installation from the official Anaconda Web site: https://docs.anaconda.com/anaconda/install/
 +
 
 +
 
 +
<br />
 +
 
 +
===Anaconda comes with a few IDE===
 +
 
 +
*Jupyter Lab
 +
*Jupyter Notebook
 +
*Spyder
 +
*Qtconsole
 +
*and others
 +
 
 +
 
 +
<br />
 +
===Anaconda Navigator===
 +
Anaconda Navigator is a GUI that helps you to easily start important applications and manage the packages in your local Anaconda installation
 +
 
 +
You can open the Anaconda Navigator from the Terminal:
 +
anaconda-navigator
 +
 
 +
 
 +
<br />
 +
 
 +
==Jupyter==
 +
Jupyter comes with Anaconda.
 +
 
 +
*It is a development environment (IDE) where we can write codes; but it also allows us to display images, and write down markdown notes.
 +
 
 +
*It is the most popular IDE in data science for exploring and analyzing data.
 +
 
 +
*Other famoues IDE for Python are Sublime Text and PyCharm.
 +
 
 +
*There is '''Jupyter Lab''' and '''Jupyter Notebook'''
 +
 
 +
 
 +
<br />
 +
===Remote connection===
 +
https://jupyter-notebook.readthedocs.io/en/stable/public_server.html
 +
 
 +
 
 +
A**1
 +
 
 +
 
 +
<syntaxhighlight lang="shell">
 +
(base) adelo@vmi346715:~/.jupyter$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mykey.key -out mycert.pem
 +
Generating a RSA private key
 +
......................................+++++
 +
....................................+++++
 +
writing new private key to 'mykey.key'
 +
-----
 +
You are about to be asked to enter information that will be incorporated
 +
into your certificate request.
 +
What you are about to enter is what is called a Distinguished Name or a DN.
 +
There are quite a few fields but you can leave some blank
 +
For some fields there will be a default value,
 +
If you enter '.', the field will be left blank.
 +
-----
 +
Country Name (2 letter code) [AU]:IE
 +
State or Province Name (full name) [Some-State]:Dublin
 +
Locality Name (eg, city) []:Dublin
 +
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
 +
Organizational Unit Name (eg, section) []:.
 +
Common Name (e.g. server FQDN or YOUR name) []:sinfronteras   
 +
Email Address []:adeloaleman@gmail.com
 +
</syntaxhighlight>
 +
 
 +
 
 +
<br />
 +
===Share Jupyter Notebook online===
 +
* '''GitHub:'''
 +
: https://docs.github.com/en/github/managing-files-in-a-repository/working-with-jupyter-notebook-files-on-github
 +
: Example: https://github.com/adeloaleman/AmazonLaptopsDashboard/blob/master/DataAnalysis/data_analysis2.ipynb
  
−
https://www.udemy.com/course/introduction-to-tweepy-python-twitter-library/
 
  
−
https://www.udemy.com/course/introduction-to-tweepy-python-twitter-library-part-2/
+
* '''Nbviewer''
 +
: https://nbviewer.jupyter.org/
 +
: Example: https://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/main/tutorial/06%20-%20Linking%20and%20Interactions.ipynb
  
−
https://www.udemy.com/course/python-bot-development/
 
  
 +
<br />
  
−
https://medium.com/@stefanobosisio1/sentiment-analysis-to-predict-political-elections-outcome-506284890132
+
===Customize Jupyter===
  
−
https://ricochet.com/565273/archives/predicting-elections-using-a-i-and-machine-learning/
 
  
−
https://link.springer.com/article/10.1007/s40092-017-0238-2
+
<br />
 +
====Themes====
 +
https://github.com/dunovank/jupyter-themes
  
−
https://pdfs.semanticscholar.org/fd8b/8e5d4ae127afc71437d39881d69d963ed80a.pdf
+
Ver el tema que muestran en esta página: https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab
  
−
https://medium.com/analytics-vidhya/twitter-sentiment-analysis-for-the-2019-election-8f7d52af1887
 
  
−
https://www.researchgate.net/publication/280623314_The_machine_learning_in_the_prediction_of_elections
+
jt  -t oceans16    -cellw 98%  -lineh 120  -fs 14  -nfs 14  -dfs 14  -ofs 14
  
  
−
https://www.udemy.com/course/data-science-natural-language-processing-in-python/
+
https://www.kaggle.com/getting-started/97540
 +
jt  -t monokai      -cellw 98%  -lineh 120  -fs 14  -nfs 14  -dfs 14  -ofs 14  -f fira  -nf ptsans  -N  -kl  -cursw 2  -cursc r  -T
  
−
https://www.udemy.com/course/sentiment-analysis-deep-learning-keras-python/
 
  
−
https://www.udemy.com/course/natural-language-processing-with-deep-learning-in-python/
+
<br />
  
 +
====Extensions====
 +
This post mention so nice extension and configuration that can be done: https://towardsdatascience.com/bringing-the-best-out-of-jupyter-notebooks-for-data-science-f0871519ca29
 +
 +
<br />
 +
=====Unofficial Jupyter Notebook Extensions=====
 +
https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/index.html
 +
 +
<span style="color: green">'''This is very important. There are very nice extensions in this package:'''</span>
 +
 +
* toc2: https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/toc2/README.html
 +
* Collapsible Headings
 +
* ... etc
 +
 +
<br />
 +
======Installation======
 +
https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/install.html
 +
 +
<span style="color: red">'''I had some issues to install it. La format indicada por defecto:'''</span>
 +
 +
pip install jupyter_contrib_nbextensions
 +
jupyter contrib nbextension install --user
 +
 +
<span style="color: red">'''A través de la forma anterior no pude instalar el paquete de forma correcta. La instalación no retornó errorres, y la extensión se mostraba en Jupyter-notebook pero no podía activar "enable" las extensiones.'''</span>
 +
 +
 +
<span style="color: red">'''Al parecer es un problema con la ubicación de la instalación. Yo estaba usando conda pero conda está presentando problemas. La instalación de los paquestes demora muchísimo y luego el paquete parece no estar disponible.'''</span>
 +
 +
 +
<span style="color: red">'''En el siguiente post encontré una solución para instalar nbextension usando pip:'''</span>
 +
https://github.com/ipython-contrib/jupyter_contrib_nbextensions/issues/1127
 +
 +
pip install --upgrade jupyter_contrib_nbextensions
 +
jupyter contrib nbextension install  --sys-prefix  --symlink
 +
 +
<span style="color: red">'''«--symlink» creo que lo usé pero no estoy completamente seguro. También realicé el --upgrade pero creo que la diferencia la hicieron las opciones --sys-prefix  --symlink'''</span>
 +
 +
 +
 +
Si no se muestra la '''Nbextensions''' tab (), try to reinstall the https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator
 +
 +
pip install jupyter_nbextensions_configurator
 +
or
 +
conda install -c conda-forge jupyter_nbextensions_configurator
 +
 +
 +
<br />
 +
 +
====CustomJS and CustonCSS files====
 +
This is a good post: https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064
 +
 +
Keyboard Shortcut Customization: https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Custom%20Keyboard%20Shortcuts.html
 +
 +
 +
<br />
 +
custom.js
 +
<syntaxhighlight lang="js">
 +
/** Mis configuraciones */
 +
 +
// This is to enable syntax highlighting for SQL code:
 +
// https://stackoverflow.com/questions/43641362/adding-syntax-highlighting-to-jupyter-notebook-cell-magic
 +
require(['notebook/js/codecell'], function(codecell) {
 +
  codecell.CodeCell.options_default.highlight_modes['magic_text/x-mssql'] = {'reg':[/^%%sql/]} ;
 +
  Jupyter.notebook.events.one('kernel_ready.Kernel', function(){
 +
  Jupyter.notebook.get_cells().map(function(cell){
 +
      if (cell.cell_type == 'code'){ cell.auto_highlight(); } }) ;
 +
  });
 +
});
 +
 +
 +
// My plain theme
 +
// This is a good post where I took some ideas to write the following fuction: https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064
 +
function plainTheme() {
 +
    var input_promp_fields = document.getElementsByClassName("prompt_container");
 +
    var text_render_fields = document.getElementsByClassName("text_cell_render");
 +
 +
    if (input_promp_fields[0].style.visibility == "collapse"){
 +
        action = "visible";
 +
        input_marginLeft = "0px";
 +
        border_top  = "3px";
 +
        prompt_width = "74px";
 +
        padding_top = "0px";
 +
        output_margin = "40px";
 +
    }else{
 +
        action = "collapse";
 +
        input_marginLeft = "74px";
 +
        border_top  = '0px';
 +
        prompt_width = "74px";
 +
        padding_top = "40px";
 +
        output_margin = "40px";
 +
    }
 +
 +
    // Si queremos usar !important debemos hacerlo de esta forma utilizando JQuery:
 +
    // https://makitweb.com/how-to-add-important-to-css-property-with-jquery/
 +
    var text_cell_fields = document.getElementsByClassName("text_cell");
 +
    $(text_cell_fields).ready(function(){
 +
        $('.input_prompt').css({
 +
            'cssText': `width: 40px !important; max-width: ${prompt_width} !important; min-width: ${prompt_width} !important;`
 +
        });
 +
    });
 +
 +
    $(document).ready(function(){
 +
        $(".prompt_container").css(
 +
            'visibility', `${action}`
 +
        );
 +
       
 +
        $(".input").css(
 +
            'padding-left', `${input_marginLeft}`
 +
        );
 +
       
 +
        $(".output_subarea").css(
 +
            'margin-left', `${output_margin}`
 +
        );
 +
                   
 +
        $('.cell').css({
 +
            'cssText': `border-top-width: ${border_top} !important; border-bottom-width: ${border_top} !important;`
 +
        });
 +
       
 +
        $(".collapsible_headings_ellipsis").css({
 +
            'cssText': `padding-top:${padding_top} !important; border-top-width: ${border_top} !important; border-bottom-width: ${border_top} !important;`
 +
        });
 +
 +
        $(".text_cell_render").css({
 +
            'cssText': `margin-left: -10px;`
 +
        });
 +
    });           
 +
}
 +
 +
Jupyter.keyboard_manager.command_shortcuts.add_shortcut('Alt-Ctrl-Q', {
 +
    help : '...',
 +
    help_index : 'zz',
 +
    handler : function (event) {
 +
        plainTheme();
 +
    return false;
 +
    }}
 +
);
 +
 +
Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Alt-Ctrl-Q', {
 +
    help : '...',
 +
    help_index : 'zz',
 +
    handler : function (event) {
 +
        plainTheme();
 +
    return false;
 +
    }}
 +
);
 +
 +
 +
// This could be very usefull. It allows to add text automatically into a cell
 +
// https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064/27
 +
Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Ctrl-Shift-J', {
 +
    help : '...',
 +
    help_index : 'zz',
 +
    handler : function (event) {
 +
        document.body.style.background = 'blue'
 +
        var target = Jupyter.notebook.get_selected_cell()
 +
        var cursor = target.code_mirror.getCursor()
 +
        var before = target.get_pre_cursor()
 +
        var after = target.get_post_cursor()
 +
        target.set_text(before + 'from IPython.core.display import display, HTML; \n\taverrrdisplay(HTML("<style>.container { width:98% !important;}</style>"))' + after)
 +
        cursor.ch += 20 // where to put your cursor
 +
        target.code_mirror.setCursor(cursor)
 +
        return false;
 +
    }}
 +
);
 +
 +
 +
// To get the real value of a css field: https://stackoverflow.com/questions/26074476/document-body-style-backgroundcolor-doesnt-work-with-external-css-style-sheet
 +
// window.getComputedStyle(document.body).backgroundColor
 +
// window.getComputedStyle(document.getElementsByClassName("input_area")[0]).backgroundColor
 +
</syntaxhighlight>
 +
 +
 +
<br />
 +
custom.css
 +
<syntaxhighlight lang="css">
 +
/*  Mis configuraciones  */
 +
 +
.container { width:98% !important; }
 +
/* document.getElementById("notebook-container").style.minWidth = "50%"; */
 +
/* document.getElementById("notebook-container").style.maxWidth = "50%"; */
 +
 +
#notebook-container {
 +
width:98% !important;
 +
}
 +
 +
.CodeMirror-gutters {
 +
background-color: transparent !important;
 +
background: transparent !important;
 +
}
 +
 +
.CodeMirror-linenumber {
 +
margin-left: -20px !important;
 +
}
 +
 +
.output_subarea {
 +
margin-left: 40px !important;
 +
}
 +
 +
#toc .fa-fw {
 +
color: blue !important;
 +
}
 +
 +
#toc .highlight_on_scroll {
 +
margin-left: -4px !important;
 +
 +
}
 +
 +
#toc {
 +
padding-left: 10px !important;
 +
}
 +
 +
/*  I have also changed the color
 +
/*  #a6e22e  by  #388bfd
 +
*  in the entire custom.css
 +
*/
 +
 +
/* I have also chenged some of the properties of the toc directly above in the code:
 +
 +
#toc-wrapper {
 +
z-index: 90;
 +
position: fixed !important;
 +
display: flex;
 +
flex-direction: column;
 +
overflow: hidden;
 +
padding: 10px;
 +
padding-top: 40px !important;
 +
border-style: solid;
 +
border-width: thin;
 +
border-right-width: medium !important;
 +
background-color: #1e1e1e !important;
 +
}
 +
#toc-wrapper.ui-draggable.ui-resizable.sidebar-wrapper {
 +
border-color: rgba(93,92,82,.25) !important;
 +
}
 +
#toc a,
 +
#navigate_menu a,
 +
.toc {
 +
color: #f8f8f0 !important;
 +
font-size: 16pt !important;
 +
}
 +
#toc li > span:hover {
 +
background-color: rgba(93,92,82,.25) !important;
 +
}
 +
#toc a:hover,
 +
#navigate_menu a:hover,
 +
.toc {
 +
color: #DAA520 !important;
 +
font-size: 16pt !important;
 +
}
 +
#toc-wrapper .toc-item-num {
 +
color: #388bfd !important;
 +
font-size: 16pt !important;
 +
}
 +
*/
 +
</syntaxhighlight>
 +
 +
 +
<br />
 +
 +
====Configurations from the Juniper notebook====
 +
 +
<syntaxhighlight lang="python3">
 +
from IPython.core.display import display, HTML;
 +
 +
display(HTML("<style>.container { width:98% !important;}</style>"<))
 +
 +
display(HTML('<style>.prompt.input_prompt{display:none !important;}</style>'))
 +
display(HTML('<style>.prompt.input_prompt{visibility: visible !important;</style>'))
 +
display(HTML('<style>.prompt.input_prompt{margin-left8kmclustering.ipynb 50px}</style>'))
 +
display(HTML('<style>.prompt.input_prompt{visibility: visible !important; width: 0px !important; min-width: 0px !important}</style>')) 
 +
 +
display(HTML('<style>.input_area{margin-left: -50px;}</style>'))
 +
display(HTML('<style>.input{margin-left: -20px;}</style>'))
 +
 +
display(HTML('<style>.output_area{margin-left: 55px}</style>'))
 +
 +
# display(HTML('<style>.cell{margin-bottom: -5px !important; margin-top: -5px !important;}</style>'))
 +
# display(HTML('<style>.code_cell{margin-bottom: -5px !important; margin-top: -5px !important;}</style>'))
 +
 +
# display(HTML('<style>.output_wrapper{margin-bottom: 0px !important; margin-top: 0px !important;}</style>'))
 +
</syntaxhighlight>
 +
 +
 +
<br />
 +
 +
===Online Jupyter===
 +
There are many sites that provides solutions to run your Jupyter Notebook in the cloud: https://www.dataschool.io/cloud-services-for-jupyter-notebook/
 +
 +
I have tried:
 +
 +
*https://cocalc.com/app
 +
 +
::https://cocalc.com/projects/595bf475-61a7-47fa-af69-ba804c3f23f9/files/?session=default
 +
::Parece bueno, pero tiene opciones que no son gratis
 +
 +
 +
*https://www.kaggle.com/
 +
 +
::https://www.kaggle.com/adeloaleman/kernel1917a91630/edit
 +
::Parece bueno pero no encontré la forma adicionar una TOC
 +
 +
 +
*https://drive.google.com
 +
 +
:*https://colab.research.google.com
 +
::Es el que estoy utilizando ahora
 +
 +
 +
<br />
 +
===Some remarks===
 +
 +
 +
<br />
 +
====Executing Terminal Commands in Jupyter Notebooks====
 +
https://support.anaconda.com/hc/en-us/articles/360023858254-Executing-Terminal-Commands-in-Jupyter-Notebooks
 +
 +
If we are in the Notebook, and we want to run a shell command rather than a notebook command we use the <code>'''!''' or '''%'''</code>
 +
 +
Try, for example:
 +
%ls
 +
!pwd
 +
 +
It's the same as if you opened up a terminal and typed it without the <code>'''!'''</code>
 +
 +
 +
<br />
 +
 +
===[[HTML presentation with Reveal.js#Creating Presentations in Jupyter Notebook with RevealJS|Creating Presentations in Jupyter Notebook with RevealJS]]===
 +
 +
 +
<br />
 +
 +
==Some of the most popular Python Data Science Libraries==
 +
 +
*NumPy
 +
*SciPy
 +
*Pandas
 +
*Seaborn
 +
*SciKit'Learn
 +
*MatplotLib
 +
*Plotly
 +
*PySpartk
 +
 +
 +
<br />
 +
 +
==[[NumPy and Pandas]]==
 +
 +
 +
<br />
 +
==[[Data Visualization with Python]]==
 +
 +
 +
<br />
 +
 +
==[[Natural Language Processing]]==
 +
 +
 +
<br />
 +
==[[Dash - Plotly]]==
 +
 +
 +
<br />
 +
==[[Scrapy]]==
 +
 +
 +
<br />
 +
==Using SQL in Jupyter==
 +
Connecting to a database in Jupyter
 +
 +
 +
https://pypi.org/project/ipython-sql/
 +
 +
https://stackoverflow.com/questions/454854/no-module-named-mysqldb
 +
 +
https://stackoverflow.com/questions/5178292/pip-install-mysql-python-fails-with-environmenterror-mysql-config-not-found
 +
 +
https://docs.kyso.io/guides/sql-interface-within-jupyterlab
 +
 +
https://www.datacamp.com/community/tutorials/sql-interface-within-jupyterlab
 +
 +
https://stackoverflow.com/questions/43641362/adding-syntax-highlighting-to-jupyter-notebook-cell-magic
 +
 +
https://www.sqlshack.com/learn-jupyter-notebooks-for-sql-server/
 +
 +
 +
Verificar las fuentes above. Creo que lo único que tuve que hacer la última vez que lo instalé fue basado en las 3 primeras sources:
 +
 +
pip install ipython-sql
 +
 +
sudo apt install default-libmysqlclient-dev
 +
 +
pip install mysqlclient
 +
 +
sudo apt-get install python3-mysqldb
  
−
==Anaconda==
 
−
Anaconda is a free and open source distribution of the Python and R programming languages for data science and machine learning related applications (large-scale data processing, predictive analytics, scientific computing), that aims to simplify package management and deployment. Package versions are managed by the package management system conda. https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)
 
  
−
===Installation===
+
Luego adding SQL syntax highlighting to Jupyter as describe above in the corrrespoinding source.
−
https://www.anaconda.com/download/#linux
 
  
−
https://linuxize.com/post/how-to-install-anaconda-on-ubuntu-18-04/
 
  
−
https://www.digitalocean.com/community/tutorials/how-to-install-the-anaconda-python-distribution-on-ubuntu-18-04
+
<br />

Latest revision as of 14:47, 11 September 2024


For a standard Python tutorial go to Python



Courses

  • Udemy - Python for Data Science and Machine Learning Bootcamp
https://www.udemy.com/course/python-for-data-science-and-machine-learning-bootcamp/



Anaconda

Anaconda is a free and open source distribution of the Python and R programming languages for data science and machine learning related applications (large-scale data processing, predictive analytics, scientific computing), that aims to simplify package management and deployment. Package versions are managed by the package management system conda. https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)

En otras palabras, Anaconda puede ser visto como un paquete (a distribution) que incluye no solo Python (or R) but many libraries that are used in Data Science, as well as its own virtual environment system. It's an "all-in-one" install that is extremely popular in data science and Machine Learning.Creating sample array for the following examples:



Installation

Installation from the official Anaconda Web site: https://docs.anaconda.com/anaconda/install/



Anaconda comes with a few IDE

  • Jupyter Lab
  • Jupyter Notebook
  • Spyder
  • Qtconsole
  • and others



Anaconda Navigator

Anaconda Navigator is a GUI that helps you to easily start important applications and manage the packages in your local Anaconda installation

You can open the Anaconda Navigator from the Terminal: 

anaconda-navigator



Jupyter

Jupyter comes with Anaconda.

  • It is a development environment (IDE) where we can write codes; but it also allows us to display images, and write down markdown notes.
  • It is the most popular IDE in data science for exploring and analyzing data.
  • Other famoues IDE for Python are Sublime Text and PyCharm.
  • There is Jupyter Lab and Jupyter Notebook



Remote connection

https://jupyter-notebook.readthedocs.io/en/stable/public_server.html


A**1 


(base) adelo@vmi346715:~/.jupyter$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mykey.key -out mycert.pem
Generating a RSA private key
......................................+++++
....................................+++++
writing new private key to 'mykey.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:IE	
State or Province Name (full name) [Some-State]:Dublin
Locality Name (eg, city) []:Dublin
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (e.g. server FQDN or YOUR name) []:sinfronteras    
Email Address []:adeloaleman@gmail.com



Share Jupyter Notebook online

  • GitHub:
https://docs.github.com/en/github/managing-files-in-a-repository/working-with-jupyter-notebook-files-on-github
Example: https://github.com/adeloaleman/AmazonLaptopsDashboard/blob/master/DataAnalysis/data_analysis2.ipynb


  • 'Nbviewer
https://nbviewer.jupyter.org/
Example: https://nbviewer.jupyter.org/github/bokeh/bokeh-notebooks/blob/main/tutorial/06%20-%20Linking%20and%20Interactions.ipynb



Customize Jupyter


Themes

https://github.com/dunovank/jupyter-themes

Ver el tema que muestran en esta página: https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab 


jt   -t oceans16     -cellw 98%   -lineh 120   -fs 14   -nfs 14   -dfs 14   -ofs 14


https://www.kaggle.com/getting-started/97540 

jt   -t monokai      -cellw 98%   -lineh 120   -fs 14   -nfs 14   -dfs 14   -ofs 14   -f fira   -nf ptsans   -N   -kl   -cursw 2   -cursc r   -T



Extensions

This post mention so nice extension and configuration that can be done: https://towardsdatascience.com/bringing-the-best-out-of-jupyter-notebooks-for-data-science-f0871519ca29


Unofficial Jupyter Notebook Extensions

https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/index.html

This is very important. There are very nice extensions in this package:


Installation

https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/install.html

I had some issues to install it. La format indicada por defecto: 

pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user

A través de la forma anterior no pude instalar el paquete de forma correcta. La instalación no retornó errorres, y la extensión se mostraba en Jupyter-notebook pero no podía activar "enable" las extensiones.


Al parecer es un problema con la ubicación de la instalación. Yo estaba usando conda pero conda está presentando problemas. La instalación de los paquestes demora muchísimo y luego el paquete parece no estar disponible.


En el siguiente post encontré una solución para instalar nbextension usando pip: https://github.com/ipython-contrib/jupyter_contrib_nbextensions/issues/1127 

pip install --upgrade jupyter_contrib_nbextensions
jupyter contrib nbextension install  --sys-prefix  --symlink

«--symlink» creo que lo usé pero no estoy completamente seguro. También realicé el --upgrade pero creo que la diferencia la hicieron las opciones --sys-prefix --symlink


Si no se muestra la Nbextensions tab (), try to reinstall the https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator 

pip install jupyter_nbextensions_configurator

or 

conda install -c conda-forge jupyter_nbextensions_configurator



CustomJS and CustonCSS files

This is a good post: https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064

Keyboard Shortcut Customization: https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Custom%20Keyboard%20Shortcuts.html 



custom.js

/** Mis configuraciones */ 

// This is to enable syntax highlighting for SQL code: 
// https://stackoverflow.com/questions/43641362/adding-syntax-highlighting-to-jupyter-notebook-cell-magic
require(['notebook/js/codecell'], function(codecell) {
  codecell.CodeCell.options_default.highlight_modes['magic_text/x-mssql'] = {'reg':[/^%%sql/]} ;
  Jupyter.notebook.events.one('kernel_ready.Kernel', function(){
  Jupyter.notebook.get_cells().map(function(cell){
      if (cell.cell_type == 'code'){ cell.auto_highlight(); } }) ;
  });
});


// My plain theme
// This is a good post where I took some ideas to write the following fuction: https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064
function plainTheme() {
    var input_promp_fields = document.getElementsByClassName("prompt_container");
    var text_render_fields = document.getElementsByClassName("text_cell_render");

    if (input_promp_fields[0].style.visibility == "collapse"){
        action = "visible";
        input_marginLeft = "0px";
        border_top  = "3px";
        prompt_width = "74px";
        padding_top = "0px";
        output_margin = "40px";
    }else{
        action = "collapse";
        input_marginLeft = "74px";
        border_top  = '0px';
        prompt_width = "74px";
        padding_top = "40px";
        output_margin = "40px";
    }

    // Si queremos usar !important debemos hacerlo de esta forma utilizando JQuery:
    // https://makitweb.com/how-to-add-important-to-css-property-with-jquery/
    var text_cell_fields = document.getElementsByClassName("text_cell");
    $(text_cell_fields).ready(function(){
        $('.input_prompt').css({
            'cssText': `width: 40px !important; max-width: ${prompt_width} !important; min-width: ${prompt_width} !important;`
        });
    });

    $(document).ready(function(){
        $(".prompt_container").css(
            'visibility', `${action}`
        );
        
        $(".input").css(
            'padding-left', `${input_marginLeft}`
        );
        
        $(".output_subarea").css(
            'margin-left', `${output_margin}`
        );
                    
        $('.cell').css({
            'cssText': `border-top-width: ${border_top} !important; border-bottom-width: ${border_top} !important;`
        });
        
        $(".collapsible_headings_ellipsis").css({
            'cssText': `padding-top:${padding_top} !important; border-top-width: ${border_top} !important; border-bottom-width: ${border_top} !important;`
        });

        $(".text_cell_render").css({
            'cssText': `margin-left: -10px;`
        });
    });            
}

Jupyter.keyboard_manager.command_shortcuts.add_shortcut('Alt-Ctrl-Q', {
    help : '...',
    help_index : 'zz',
    handler : function (event) {
        plainTheme();
    return false;
    }}
);

Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Alt-Ctrl-Q', {
    help : '...',
    help_index : 'zz',
    handler : function (event) {
        plainTheme();
    return false;
    }}
);


// This could be very usefull. It allows to add text automatically into a cell
// https://forums.fast.ai/t/jupyter-notebook-enhancements-tips-and-tricks/17064/27
Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Ctrl-Shift-J', {
    help : '...',
    help_index : 'zz',
    handler : function (event) {
        document.body.style.background = 'blue'
        var target = Jupyter.notebook.get_selected_cell()
        var cursor = target.code_mirror.getCursor()
        var before = target.get_pre_cursor()
        var after = target.get_post_cursor()
        target.set_text(before + 'from IPython.core.display import display, HTML; \n\taverrrdisplay(HTML("<style>.container { width:98% !important;}</style>"))' + after)
        cursor.ch += 20 // where to put your cursor
        target.code_mirror.setCursor(cursor)
        return false;
    }}
);


// To get the real value of a css field: https://stackoverflow.com/questions/26074476/document-body-style-backgroundcolor-doesnt-work-with-external-css-style-sheet
// window.getComputedStyle(document.body).backgroundColor
// window.getComputedStyle(document.getElementsByClassName("input_area")[0]).backgroundColor




custom.css

/*  Mis configuraciones  */

.container { width:98% !important; }
/* document.getElementById("notebook-container").style.minWidth = "50%"; */
/* document.getElementById("notebook-container").style.maxWidth = "50%"; */

#notebook-container {
 width:98% !important;
}

.CodeMirror-gutters {
 background-color: transparent !important;
 background: transparent !important;
}

.CodeMirror-linenumber {
 margin-left: -20px !important;
}

.output_subarea {
 margin-left: 40px !important;
}

#toc .fa-fw {
 color: blue !important;
}

#toc .highlight_on_scroll {
 margin-left: -4px !important;
 
}

#toc {
 padding-left: 10px !important;
}

/*  I have also changed the color
/*  #a6e22e   by   #388bfd 
 *  in the entire custom.css
 */

/* I have also chenged some of the properties of the toc directly above in the code: 

#toc-wrapper {
 z-index: 90;
 position: fixed !important;
 display: flex;
 flex-direction: column;
 overflow: hidden;
 padding: 10px;
 padding-top: 40px !important;
 border-style: solid;
 border-width: thin;
 border-right-width: medium !important;
 background-color: #1e1e1e !important;
}
#toc-wrapper.ui-draggable.ui-resizable.sidebar-wrapper {
 border-color: rgba(93,92,82,.25) !important;
}
#toc a,
#navigate_menu a,
.toc {
 color: #f8f8f0 !important;
 font-size: 16pt !important;
}
#toc li > span:hover {
 background-color: rgba(93,92,82,.25) !important;
}
#toc a:hover,
#navigate_menu a:hover,
.toc {
 color: #DAA520 !important;
 font-size: 16pt !important;
}
#toc-wrapper .toc-item-num {
 color: #388bfd !important;
 font-size: 16pt !important;
}
*/



Configurations from the Juniper notebook

from IPython.core.display import display, HTML; 

display(HTML("<style>.container { width:98% !important;}</style>"<))

display(HTML('<style>.prompt.input_prompt{display:none !important;}</style>'))
display(HTML('<style>.prompt.input_prompt{visibility: visible !important;</style>'))
display(HTML('<style>.prompt.input_prompt{margin-left8kmclustering.ipynb 50px}</style>'))
display(HTML('<style>.prompt.input_prompt{visibility: visible !important; width: 0px !important; min-width: 0px !important}</style>'))  

display(HTML('<style>.input_area{margin-left: -50px;}</style>'))
display(HTML('<style>.input{margin-left: -20px;}</style>'))

display(HTML('<style>.output_area{margin-left: 55px}</style>'))

# display(HTML('<style>.cell{margin-bottom: -5px !important; margin-top: -5px !important;}</style>'))
# display(HTML('<style>.code_cell{margin-bottom: -5px !important; margin-top: -5px !important;}</style>'))

# display(HTML('<style>.output_wrapper{margin-bottom: 0px !important; margin-top: 0px !important;}</style>'))



Online Jupyter

There are many sites that provides solutions to run your Jupyter Notebook in the cloud: https://www.dataschool.io/cloud-services-for-jupyter-notebook/

I have tried:

https://cocalc.com/projects/595bf475-61a7-47fa-af69-ba804c3f23f9/files/?session=default
Parece bueno, pero tiene opciones que no son gratis


https://www.kaggle.com/adeloaleman/kernel1917a91630/edit
Parece bueno pero no encontré la forma adicionar una TOC


Es el que estoy utilizando ahora



Some remarks


Executing Terminal Commands in Jupyter Notebooks

https://support.anaconda.com/hc/en-us/articles/360023858254-Executing-Terminal-Commands-in-Jupyter-Notebooks

If we are in the Notebook, and we want to run a shell command rather than a notebook command we use the ! or %

Try, for example: 

%ls 
!pwd

It's the same as if you opened up a terminal and typed it without the !



Creating Presentations in Jupyter Notebook with RevealJS


Some of the most popular Python Data Science Libraries

  • NumPy
  • SciPy
  • Pandas
  • Seaborn
  • SciKit'Learn
  • MatplotLib
  • Plotly
  • PySpartk



NumPy and Pandas


Data Visualization with Python


Natural Language Processing


Dash - Plotly


Scrapy


Using SQL in Jupyter

Connecting to a database in Jupyter


https://pypi.org/project/ipython-sql/

https://stackoverflow.com/questions/454854/no-module-named-mysqldb

https://stackoverflow.com/questions/5178292/pip-install-mysql-python-fails-with-environmenterror-mysql-config-not-found

https://docs.kyso.io/guides/sql-interface-within-jupyterlab

https://www.datacamp.com/community/tutorials/sql-interface-within-jupyterlab

https://stackoverflow.com/questions/43641362/adding-syntax-highlighting-to-jupyter-notebook-cell-magic

https://www.sqlshack.com/learn-jupyter-notebooks-for-sql-server/


Verificar las fuentes above. Creo que lo único que tuve que hacer la última vez que lo instalé fue basado en las 3 primeras sources: 

pip install ipython-sql

sudo apt install default-libmysqlclient-dev

pip install mysqlclient

sudo apt-get install python3-mysqldb


Luego adding SQL syntax highlighting to Jupyter as describe above in the corrrespoinding source.



Retrieved from "http://wiki.sinfronteras.ws/index.php?title=Python_for_Data_Science&oldid=25684"