昇思MindSpore

昇思MindSpore

华为开源的自研AI深度学习框架

访问官网 >

昇思MindSpore是什么

昇思MindSpore是华为推出的适用端边云场景的新型开源全场景深度学习框架，昇思MindSpore具备强大的分布式训练能力，内置多种并行策略，简化大模型开发。昇思MindSpore与昇腾处理器深度适配，充分发挥硬件性能，缩短训练时间并提升推理效率。昇思MindSpore支持AI与高性能计算（HPC）融合，满足AI for Science场景需求。昇思MindSpore生态丰富，提供开源项目、案例和SOTA模型，方便开发者快速上手和应用。

昇思MindSpore

昇思MindSpore的主要功能

  • 全场景AI框架：昇思MindSpore支持云、边缘和端侧的快速部署，适用多种硬件平台，如昇腾处理器、GPU、CPU等。
  • 分布式训练：内置多种并行能力，提供简单易用的大模型分布式策略配置接口，支持高效的大模型训练。
  • AI4S融合计算框架：支持AI + HPC全流程可编程，满足AI for Science场景的异构并行加速需求。
  • 硬件潜能发挥：动静统一编程，最佳匹配昇腾处理器，帮助开发者缩短训练时间，提升推理性能。
  • 安装与使用：提供详细的安装指南和命令，支持多种硬件平台。
  • 云平台支持：提供云平台支持，帮助用户快速创建和部署模型，管理全周期AI工作流。
  • 生态资源：提供开源项目、案例集合、SOTA模型及其衍生工具，支持多种领域套件（如NLP、CV、Audio等）。

如何使用昇思MindSpore

  • 环境准备：
    • 硬件环境：昇思MindSpore支持多种硬件平台，包括昇腾处理器、GPU（如NVIDIA系列）、CPU（如Intel x86、Arm架构）等。
    • 软件环境：
      • 操作系统：推荐使用Linux（如Ubuntu 18.04或CentOS 7.6及以上版本），也支持Windows和macOS。
      • Python环境：推荐使用Python 3.7及以上版本。
  • 安装MindSpore：
    • 安装方式：根据使用的硬件平台，选择合适的安装命令。以下是一些常见的安装方式：
      • 安装CPU版本：
pip <span class="token function">install</span> mindspore
      • 安装GPU版本（以CUDA 11.1为例）：
pip <span class="token function">install</span> mindspore-gpu<span class="token operator">==</span><span class="token number">1.10</span>.0
      • 安装昇腾版本：
pip <span class="token function">install</span> https://ms-release.obs.cn-north-4.myhuaweicloud.com/2.6.0/MindSpore/unified/aarch64/mindspore-2.6.0-cp39-cp39-linux_aarch64.whl --trusted-host ms-release.obs.cn-north-4.myhuaweicloud.com <span class="token parameter variable">-i</span> https://pypi.tuna.tsinghua.edu.cn/simple
    • 验证安装：安装完成后，运行以下命令验证MindSpore是否安装成功：
<span class="token keyword">import</span> mindspore<span class="token keyword">print</span><span class="token punctuation">(</span>mindspore<span class="token punctuation">.</span>__version__<span class="token punctuation">)</span>
  • 快速入门：
    • 数据处理：框架提供高效的数据处理Pipeline，支持多种数据集加载和预处理操作。
<span class="token keyword">import</span> mindspore<span class="token punctuation">.</span>dataset <span class="token keyword">as</span> ds<span class="token keyword">from</span> mindspore<span class="token punctuation">.</span>dataset<span class="token punctuation">.</span>transforms <span class="token keyword">import</span> Compose<span class="token punctuation">,</span> ToTensor<span class="token punctuation">,</span> Normalize<span class="token comment"># 加载MNIST数据集</span>dataset <span class="token operator">=</span> ds<span class="token punctuation">.</span>MnistDataset<span class="token punctuation">(</span><span class="token string">"path/to/mnist_dataset"</span><span class="token punctuation">)</span>transforms <span class="token operator">=</span> Compose<span class="token punctuation">(</span><span class="token punctuation">[</span>ToTensor<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Normalize<span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token number">0.1307</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token number">0.3081</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>dataset <span class="token operator">=</span> dataset<span class="token punctuation">.</span><span class="token builtin">map</span><span class="token punctuation">(</span>operations<span class="token operator">=</span>transforms<span class="token punctuation">,</span> input_columns<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"image"</span><span class="token punctuation">]</span><span class="token punctuation">)</span>dataset <span class="token operator">=</span> dataset<span class="token punctuation">.</span>batch<span class="token punctuation">(</span>batch_size<span class="token operator">=</span><span class="token number">64</span><span class="token punctuation">)</span>
    • 构建模型：框架提供丰富的API，支持动态图和静态图编程。
<span class="token keyword">import</span> mindspore<span class="token punctuation">.</span>nn <span class="token keyword">as</span> nn<span class="token keyword">import</span> mindspore<span class="token punctuation">.</span>ops <span class="token keyword">as</span> ops<span class="token keyword">class</span> <span class="token class-name">LeNet5</span><span class="token punctuation">(</span>nn<span class="token punctuation">.</span>Cell<span class="token punctuation">)</span><span class="token punctuation">:</span>    <span class="token keyword">def</span> <span class="token function">__init__</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span>        <span class="token builtin">super</span><span class="token punctuation">(</span>LeNet5<span class="token punctuation">,</span> self<span class="token punctuation">)</span><span class="token punctuation">.</span>__init__<span class="token punctuation">(</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>conv1 <span class="token operator">=</span> nn<span class="token punctuation">.</span>Conv2d<span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">,</span> pad_mode<span class="token operator">=</span><span class="token string">'valid'</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>conv2 <span class="token operator">=</span> nn<span class="token punctuation">.</span>Conv2d<span class="token punctuation">(</span><span class="token number">6</span><span class="token punctuation">,</span> <span class="token number">16</span><span class="token punctuation">,</span> <span class="token number">5</span><span class="token punctuation">,</span> pad_mode<span class="token operator">=</span><span class="token string">'valid'</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>fc1 <span class="token operator">=</span> nn<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">16</span> <span class="token operator">*</span> <span class="token number">5</span> <span class="token operator">*</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token number">120</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>fc2 <span class="token operator">=</span> nn<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">120</span><span class="token punctuation">,</span> <span class="token number">84</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>fc3 <span class="token operator">=</span> nn<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">84</span><span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>relu <span class="token operator">=</span> nn<span class="token punctuation">.</span>ReLU<span class="token punctuation">(</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>max_pool2d <span class="token operator">=</span> nn<span class="token punctuation">.</span>MaxPool2d<span class="token punctuation">(</span>kernel_size<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">)</span>        self<span class="token punctuation">.</span>flatten <span class="token operator">=</span> ops<span class="token punctuation">.</span>Flatten<span class="token punctuation">(</span><span class="token punctuation">)</span>    <span class="token keyword">def</span> <span class="token function">construct</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> x<span class="token punctuation">)</span><span class="token punctuation">:</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>max_pool2d<span class="token punctuation">(</span>self<span class="token punctuation">.</span>relu<span class="token punctuation">(</span>self<span class="token punctuation">.</span>conv1<span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>max_pool2d<span class="token punctuation">(</span>self<span class="token punctuation">.</span>relu<span class="token punctuation">(</span>self<span class="token punctuation">.</span>conv2<span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>flatten<span class="token punctuation">(</span>x<span class="token punctuation">)</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>relu<span class="token punctuation">(</span>self<span class="token punctuation">.</span>fc1<span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">)</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>relu<span class="token punctuation">(</span>self<span class="token punctuation">.</span>fc2<span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">)</span>        x <span class="token operator">=</span> self<span class="token punctuation">.</span>fc3<span class="token punctuation">(</span>x<span class="token punctuation">)</span>        <span class="token keyword">return</span> xmodel <span class="token operator">=</span> LeNet5<span class="token punctuation">(</span><span class="token punctuation">)</span>
    • 训练模型：框架提供Model类，简化训练和评估流程。
<span class="token keyword">from</span> mindspore<span class="token punctuation">.</span>train <span class="token keyword">import</span> Model<span class="token keyword">from</span> mindspore<span class="token punctuation">.</span>nn <span class="token keyword">import</span> SoftmaxCrossEntropyWithLogits<span class="token punctuation">,</span> Momentum<span class="token keyword">from</span> mindspore<span class="token punctuation">.</span>train<span class="token punctuation">.</span>callback <span class="token keyword">import</span> ModelCheckpoint<span class="token punctuation">,</span> CheckpointConfig<span class="token comment"># 定义损失函数和优化器</span>loss_fn <span class="token operator">=</span> SoftmaxCrossEntropyWithLogits<span class="token punctuation">(</span>sparse<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> reduction<span class="token operator">=</span><span class="token string">'mean'</span><span class="token punctuation">)</span>optimizer <span class="token operator">=</span> Momentum<span class="token punctuation">(</span>model<span class="token punctuation">.</span>trainable_params<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> learning_rate<span class="token operator">=</span><span class="token number">0.01</span><span class="token punctuation">,</span> momentum<span class="token operator">=</span><span class="token number">0.9</span><span class="token punctuation">)</span><span class="token comment"># 创建Model实例</span>model <span class="token operator">=</span> Model<span class="token punctuation">(</span>model<span class="token punctuation">,</span> loss_fn<span class="token operator">=</span>loss_fn<span class="token punctuation">,</span> optimizer<span class="token operator">=</span>optimizer<span class="token punctuation">,</span> metrics<span class="token operator">=</span><span class="token punctuation">{</span><span class="token string">'accuracy'</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token comment"># 设置保存检查点的配置</span>config_ck <span class="token operator">=</span> CheckpointConfig<span class="token punctuation">(</span>save_checkpoint_steps<span class="token operator">=</span><span class="token number">1875</span><span class="token punctuation">,</span> keep_checkpoint_max<span class="token operator">=</span><span class="token number">10</span><span class="token punctuation">)</span>ckpoint_cb <span class="token operator">=</span> ModelCheckpoint<span class="token punctuation">(</span>prefix<span class="token operator">=</span><span class="token string">"lenet"</span><span class="token punctuation">,</span> config<span class="token operator">=</span>config_ck<span class="token punctuation">)</span><span class="token comment"># 开始训练</span>model<span class="token punctuation">.</span>train<span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span> dataset<span class="token punctuation">,</span> callbacks<span class="token operator">=</span><span class="token punctuation">[</span>ckpoint_cb<span class="token punctuation">]</span><span class="token punctuation">)</span>
    • 评估模型：训练完成后，用测试集评估模型性能。
<span class="token comment"># 加载测试数据集</span>test_dataset <span class="token operator">=</span> ds<span class="token punctuation">.</span>MnistDataset<span class="token punctuation">(</span><span class="token string">"path/to/mnist_test_dataset"</span><span class="token punctuation">)</span>test_dataset <span class="token operator">=</span> test_dataset<span class="token punctuation">.</span><span class="token builtin">map</span><span class="token punctuation">(</span>operations<span class="token operator">=</span>transforms<span class="token punctuation">,</span> input_columns<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"image"</span><span class="token punctuation">]</span><span class="token punctuation">)</span>test_dataset <span class="token operator">=</span> test_dataset<span class="token punctuation">.</span>batch<span class="token punctuation">(</span>batch_size<span class="token operator">=</span><span class="token number">64</span><span class="token punctuation">)</span><span class="token comment"># 评估模型</span>acc <span class="token operator">=</span> model<span class="token punctuation">.</span><span class="token builtin">eval</span><span class="token punctuation">(</span>test_dataset<span class="token punctuation">)</span><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Accuracy: </span><span class="token interpolation"><span class="token punctuation">{</span>acc<span class="token punctuation">[</span><span class="token string">'accuracy'</span><span class="token punctuation">]</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>

昇思MindSpore的应用场景

  • 智能交通与自动驾驶：在自动驾驶车辆和智能交通系统中，用在实时目标检测、路径规划和交通流量管理，提升交通效率和安全性。
  • 工业自动化与智能制造：部署在工业生产线上，用在实时质量检测和设备故障预测，优化生产流程，提高生产效率。
  • 医疗健康与医学影像：用在医学影像分析和智能健康监测设备，辅助医生诊断疾病，提升医疗服务水平。
  • 科学研究与工程仿真：结合AI4S套件，加速电磁仿真、流体仿真和分子动力学模拟等科学计算任务，助力科研创新。
  • 企业数据分析与决策支持：用在企业客户行为分析、市场趋势预测和风险评估，帮助企业做出更精准的决策。