Sep 15, 2019 Am mainly using power shell to manage Microsoft Azure resources. It’s an open-source and cross-platform package. Powershell can also be used for automation and configuration management. Here we explain about installing to on macOS Mojave ( Version 10.14.6) and Ubuntu 18.04.3 LTS. Installation step for MacOS. Sep 25, 2019 # azure config for mac os - Microsoft has indicated # they will patch this issue: vmImage: macOS-10.14: strategy: maxParallel: 2: matrix: Python36: PYTHONVERSION: ' 3.6 '. # wait until after dev build of NumPy to pip # install matplotlib to avoid pip install of older numpy - script: python -m pip install matplotlib: displayName. The current version of the Azure CLI is 2.3.1. For information about the latest release, see the release notes. To find your installed version and see if you need to update, run az -version. Install with Homebrew. Homebrew is the easiest way to manage your CLI install. It provides convenient ways to install, update, and uninstall.
-->Important
This overview applies to the stable and experimental versions of this SDK. To change versions, use the dropdown in the upper left.
This article is a guide for different installation options for the SDK.
Default install
The default installation covers a large number of use-cases and will not install any unnecessary native dependencies in your environment. The following native package structure will be installed via the azureml-sdk
package without any extra components:
azureml-sdk
azureml-train
azureml-pipeline
Note
When submitting runs, the SDK by default installs the azureml-defaults
package to the environment where the run is executed. azureml-defaults contains the azureml-core and applicationinsights packages required for tasks such as logging metrics, uploading artifacts, and accessing data stores from within the run.
To install the default packages, run the following command.
Advanced install (extras)
The SDK contains several optional components (extras
) that will only be installed if specified. These include dependencies that aren't required for all use-cases, so they are not included in the default installation in order to avoid bloating the environment. The following table outlines optional components and their use-cases.
Additional package installs (extras ) | Use-cases and installed packages |
---|---|
accel-models | Installs azureml-accel-models . Accelerates deep neural networks on FPGAs with the Azure ML Hardware Accelerated Models Service. |
automl | Installs azureml-train-automl and other required dependencies. Provides classes for building and running automated machine learning experiments. The automl extra also installs common data science packages including pandas , numpy , and scikit-learn . See the additional use-case guidance for more information on working with automl . |
contrib | Installs azureml-contrib-* packages, which include experimental functionality or preview features. |
databricks | Installs non-native packages to ensure compatibility when working within an Azure Databricks environment. This extra cannot be combined with other components. See the additional use-case guidance for more information on using the SDK in an Azure Databricks environment. |
explain | Installs azureml-explain-model and other required dependencies. Includes classes for understanding detailed feature importance in automated model tuning. |
interpret | Installs azureml-interpret used for model interpretability, including feature and class importance for blackbox and whitebox models. |
notebooks | Installs azureml-widgets and other required dependencies. Provides support for interactive widgets in a Jupyter notebook environment. This is unnecessary to install if you aren't running in a Jupyter notebook (ex. if you are building in PyCharm), or if you don't need widgets enabled. |
services | Installs azureml-contrib-services . Provides functionality for scoring scripts to request raw HTTP access. |
tensorboard | Installs azureml-tensorboard . Provides classes and methods for exporting experiment run history and launching TensorBoard for visualizing experiment performance and structure. |
Install with extras
To install the SDK with extras
, append the component name(s) in brackets onto the default install. For example, the following command installs the SDK with the explain
and automl
variants.
Experimental version
To access pre-release version (experimental version), specify the version during installation using pre-release semantics such as:
- X.YbN # Beta release – Preview channel
- X.YrcN # Release Candidate -- PyPi
To install the experimental version of the Azure Machine Learning SDK for Python, specify the --pre
flag to the pip install
such as: $ pip install --pre azureml-sdk
Custom install
If you want to run a custom install and manually manage the dependencies in your environment, you can individually install any package in the SDK.
Note
If you get a message that PyYAML can't be uninstalled, use the following command instead:
pip install --upgrade azureml-sdk[explain,automl] --ignore-installed PyYAML
Starting with macOS Catalina, zsh (Z shell) is the default login shell and interactive shell. In zsh, use the following command which escapes brackets with ' (backslash):
pip install --upgrade azureml-sdk[explain,automl]
Check version
Run the following code to verify your SDK version. This check may be necessary for verifying compatibility with certain tutorials and code samples.
To learn more about how to configure your development environment for Azure Machine Learning service, see Configure your development environment.
Additional use-case guidance
If your use-case is described below, note the guidance and any recommended actions.
Use-case | Guidance |
---|---|
Using the automl extra | Install the SDK in a 64-bit Python environment. A 64-bit environment is required because of a dependency on the LightGBM framework. |
Using Azure Databricks | In the Azure Databricks environment, use the library sources detailed in this guide for installing the SDK. Also, see these tips for further information on working with Azure Machine Learning SDK for Python on Azure Databricks. |
Using Azure Data Science Virtual Machine | Azure Data Science Virtual Machines created after September 27, 2018 come with the Python SDK preinstalled. |
Running Azure Machine Learning tutorials or notebooks | If you are using an older version of the SDK than the one mentioned in the tutorial or notebook, you should upgrade your SDK. Some functionality in the tutorials and notebooks may require additional Python packages such as matplotlib , sicikit-learn , or pandas . Instructions in each tutorial and notebook will show you which packages are required. |
Next steps
Try these next steps to learn how to use the Azure Machine Learning service SDK for Python:
- Read the overview to learn about key classes and design patterns with code samples.
- Follow this tutorial to begin creating experiments and models.
Important
Microsoft has deprecated the Python extensions for App Service on Windows as described in this article in favor of a direct deployment to App Service on Linux.
Azure App Service is a platform-as-a-service offering for web apps, whether they are sites accessed through a browser, REST APIs used by your own clients, or event-triggered processing. App Service fully supports using Python to implement apps.
Customizable Python support for Azure App Service is provided as a set of App Service site extensions that each contain a specific version of the Python runtime. You can then install any desired packages directly into that environment, as described in this article. By customizing the environment in the App Service itself, you don't need to maintain packages in your web app projects or upload them with the app code.
Tip
Although App Service by default has Python 2.7 and Python 3.4 installed in root folders on the server, you cannot customize or install packages in these environments, nor should you depend on their presence. You should instead rely on a site extension that you control, as described in this article.
Choose a Python version through the Azure portal
Create an App Service for your web app on the Azure portal.
On the App Service's page, scroll to the Development Tools section, select Extensions, then select + Add.
Scroll down in the list to the extension that contains the version of Python you want:
Tip
If you need an older version of Python and don't see it listed in the site extensions, you can still install it through the Azure Resource Manager as described in the next section.
Select the extension, accept the legal terms, then select OK.
A notification appears in the portal when installation is complete.
Choose a Python version through the Azure Resource Manager
If you are deploying an App Service with an Azure Resource Manager template, add the site extension as a resource. Specifically, the extension appears as a nested resource (a resources
object under resources
) with the type siteextensions
and the name from siteextensions.net.
For example, after adding a reference to python361x64
(Python 3.6.1 x64), your template may look like the following (some properties omitted):
Set web.config to point to the Python interpreter
After installing the site extension (through either the portal or an Azure Resource Manager template), you next point your app's web.config file to the Python interpreter. The web.config file instructs the IIS (7+) web server running on App Service about how it should handle Python requests through either HttpPlatform (recommended) or FastCGI.
Begin by finding the full path to the site extension's python.exe, then create and modify the appropriate web.config file.
Find the path to python.exe
A Python site extension is installed on the server under d:home in a folder appropriate to the Python version and architecture (except in the case of a few older versions). For example, Python 3.6.1 x64 is installed in d:homepython361x64. The full path to the Python interpreter is then d:homepython361x64python.exe.
To see the specific path on your App Service, select Extensions on the App Service page, then select the extension in the list.
This action opens the extension's description page containing the path:
If you have trouble seeing the path for the extension, you can find it manually using the console:
- On your App Service page, select the Development Tools > Console.
- Enter the command
ls ../home
ordir ..home
to see the top-level extensions folders, such as Python361x64. - Enter a command like
ls ../home/python361x64
ordir ..homepython361x64
to verify that it contains python.exe and other interpreter files.
Configure the HttpPlatform handler
The HttpPlatform module passes socket connections directly to a standalone Python process. This pass-through allows you to run any web server you like, but requires a startup script that runs a local web server. You specify the script in the <httpPlatform>
element of web.config, where the processPath
attribute points to the site extension's Python interpreter and the arguments
attribute points to your script and any arguments you want to provide:
The HTTP_PLATFORM_PORT
environment variable shown here contains the port that your local server should listen on for connections from localhost. This example also shows how to create another environment variable, if desired, in this case SERVER_PORT
.
Configure the FastCGI handler
FastCGI is an interface that works at the request level. IIS receives incoming connections and forwards each request to a WSGI app running in one or more persistent Python processes. The wfastcgi package is pre-installed and configured with each Python site extension, so you can easily enable it by including the code in web.config like what's shown below for a web app based on the Bottle framework. Note that the full paths to python.exe and wfastcgi.py are placed in the PythonHandler
key:
Install Numpy Windows 10
The <appSettings>
defined here are available to your app as environment variables:
Install Numpy In Microsoft Azure On Mac Windows 10
- The value for
PYTHONPATH
may be freely extended but must include the root of your app. WSGI_HANDLER
must point to a WSGI app importable from your app.WSGI_LOG
is optional but recommended for debugging your app.
See Publish to Azure for additional details on web.config contents for Bottle, Flask, and Django web apps.
Install packages
The Python interpreter installed through a site extension is only one piece of your Python environment. You likely need to install different packages in that environment as well.
To install packages directly in the server environment, use one of the following methods:
Methods | Usage |
---|---|
Azure App Service Kudu console | Installs packages interactively. Packages must be pure Python or must publish wheels. |
Kudu REST API | Can be used to automate package installation. Packages must be pure Python or must publish wheels. |
Bundle with app | Install packages directly into your project and then deploy them to App Service as if they were part of your app. Depending on how many dependencies you have and how frequently you update them, this method may be the easiest way to get a working deployment going. Be advised that libraries must match the version of Python on the server, otherwise you see obscure errors after deployment. That said, because the versions of Python in the App Service site extensions are exactly the same as those versions released on python.org, you can easily obtain a compatible version for local development. |
Virtual environments | Not supported. Instead, use bundling and set the PYTHONPATH environment variable to point to the location of the packages. |
Azure App Service Kudu console
The Kudu console gives you direct, elevated command-line access to the App Service server and its file system. This is both a valuable debugging tool and allows for CLI operations such as installing packages.
Open Kudu from your App Service page on the Azure portal by selecting Development Tools > Advanced Tools, then selecting Go. This action navigates to a URL that's the same as your base App Service URL except with
.scm
inserted. For example, if your base URL ishttps://vspython-test.azurewebsites.net/
then Kudu is onhttps://vspython-test.scm.azurewebsites.net/
(which you can bookmark):Select Debug console > CMD to open the console, in which you can navigate into your Python installation and see what libraries are already there.
To install a single package:
a. Navigate to the folder of the Python installation where you want to install the package, such as d:homepython361x64.
b. Use
python.exe -m pip install <package_name>
to install a package.If you've deployed a requirements.txt for your app to the server already, install all those requirements as follows:
a. Navigate to the folder of the Python installation where you want to install the package, such as d:homepython361x64.
b. Run the command
python.exe -m pip install --upgrade -r d:homesitewwwrootrequirements.txt
.Using requirements.txt is recommended because it's easy to reproduce your exact package set both locally and on the server. Just remember to visit the console after deploying any changes to requirements.txt and run the command again.
Note
There's no C compiler on App Service, so you need to install the wheel for any packages with native extension modules. Many popular packages provide their own wheels. For packages that don't, use pip wheel <package_name>
on your local development computer and then upload the wheel to your site. For an example, see Manage required packages with requirements.txt.
Kudu REST API
Instead of using the Kudu console through the Azure portal, you can run commands remotely through the Kudu REST API by posting the command to https://yoursite.scm.azurewebsites.net/api/command
. For example, to install the bottle
package, post the following JSON to /api/command
:
For information about commands and authentication, see the Kudu documentation.
You can also see credentials using the az webapp deployment list-publishing-profiles
command through the Azure CLI (see az webapp deployment). A helper library for posting Kudu commands is available on GitHub.