更新ONNX CPU lib
parent
7b7f1aea62
commit
e6ca6c2b7e
@ -0,0 +1,4 @@
|
||||
| FileName | Status | FileSize | TotalTime(sec) | Upload(sec) | Submit(sec) | SignWait(sec) | Retry Count |
|
||||
|---------------------------|--------|----------|----------------|-------------|-------------|---------------|-------------|
|
||||
| onnxruntime.dll | Pass | 7.99MB | 73.78 | 1.01 | 0.46 | 72.3 | 0 |
|
||||
| time_providers_shared.dll | Pass | 11.5KB | 53.29 | 0.51 | 0.34 | 51.81 | 0 |
|
||||
@ -0,0 +1 @@
|
||||
b353e0b41d588605958b03f9a223d10a2fbeb514
|
||||
@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) Microsoft Corporation
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@ -0,0 +1,21 @@
|
||||
# Privacy
|
||||
|
||||
## Data Collection
|
||||
The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
|
||||
|
||||
***
|
||||
|
||||
### Private Builds
|
||||
No data collection is performed when using your private builds built from source code.
|
||||
|
||||
### Official Builds
|
||||
ONNX Runtime does not maintain any independent telemetry collection mechanisms outside of what is provided by the platforms it supports. However, where applicable, ONNX Runtime will take advantage of platform-supported telemetry systems to collect trace events with the goal of improving product quality.
|
||||
|
||||
Currently telemetry is only implemented for Windows builds and is turned **ON** by default in the official builds distributed in their respective package management repositories ([see here](../README.md#binaries)). This may be expanded to cover other platforms in the future. Data collection is implemented via 'Platform Telemetry' per vendor platform providers (see [telemetry.h](../onnxruntime/core/platform/telemetry.h)).
|
||||
|
||||
#### Technical Details
|
||||
The Windows provider uses the [TraceLogging](https://docs.microsoft.com/en-us/windows/win32/tracelogging/trace-logging-about) API for its implementation. This enables ONNX Runtime trace events to be collected by the operating system, and based on user consent, this data may be periodically sent to Microsoft servers following GDPR and privacy regulations for anonymity and data access controls.
|
||||
|
||||
Windows ML and onnxruntime C APIs allow Trace Logging to be turned on/off (see [API pages](../README.md#api-documentation) for details).
|
||||
For information on how to enable and disable telemetry, see [C API: Telemetry](./C_API.md#telemetry).
|
||||
There are equivalent APIs in the C#, Python, and Java language bindings as well.
|
||||
@ -0,0 +1,52 @@
|
||||
<p align="center"><img width="50%" src="docs/images/ONNX_Runtime_logo_dark.png" /></p>
|
||||
|
||||
**ONNX Runtime is a cross-platform inference and training machine-learning accelerator**.
|
||||
|
||||
**ONNX Runtime inference** can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. [Learn more →](https://www.onnxruntime.ai/docs/#onnx-runtime-for-inferencing)
|
||||
|
||||
**ONNX Runtime training** can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. [Learn more →](https://www.onnxruntime.ai/docs/#onnx-runtime-for-training)
|
||||
|
||||
|
||||
## Get Started
|
||||
|
||||
**General Information**: [onnxruntime.ai](https://onnxruntime.ai)
|
||||
|
||||
**Usage documention and tutorials**: [onnxruntime.ai/docs](https://onnxruntime.ai/docs)
|
||||
|
||||
**Companion sample repositories**:
|
||||
- ONNX Runtime Inferencing: [microsoft/onnxruntime-inference-examples](https://github.com/microsoft/onnxruntime-inference-examples)
|
||||
- ONNX Runtime Training: [microsoft/onnxruntime-training-examples](https://github.com/microsoft/onnxruntime-training-examples)
|
||||
|
||||
|
||||
## Build Pipeline Status
|
||||
|System|CPU|GPU|EPs|
|
||||
|---|---|---|---|
|
||||
|Windows|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=9)|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=10)|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=47)|
|
||||
|Linux|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=11)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=64)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=110)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=78)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=86)|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=12)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=45)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=140)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=84)|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=55)|
|
||||
|Mac|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=13)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=65)|||
|
||||
|Android|||[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=53)|
|
||||
|iOS|||[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=134)|
|
||||
|WebAssembly|||[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=161)|
|
||||
|
||||
|
||||
## Data/Telemetry
|
||||
|
||||
Windows distributions of this project may collect usage data and send it to Microsoft to help improve our products and services. See the [privacy statement](docs/Privacy.md) for more details.
|
||||
|
||||
## Contributions and Feedback
|
||||
|
||||
We welcome contributions! Please see the [contribution guidelines](CONTRIBUTING.md).
|
||||
|
||||
For feature requests or bug reports, please file a [GitHub Issue](https://github.com/Microsoft/onnxruntime/issues).
|
||||
|
||||
For general discussion or questions, please use [GitHub Discussions](https://github.com/microsoft/onnxruntime/discussions).
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
|
||||
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
|
||||
or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the [MIT License](LICENSE).
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1 @@
|
||||
1.13.1
|
||||
@ -0,0 +1,19 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#include "onnxruntime_c_api.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/**
|
||||
* \param use_arena zero: false. non-zero: true.
|
||||
*/
|
||||
ORT_EXPORT
|
||||
ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_CPU, _In_ OrtSessionOptions* options, int use_arena)
|
||||
ORT_ALL_ARGS_NONNULL;
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,27 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
/*
|
||||
* This file defines RunOptions Config Keys and format of the Config Values.
|
||||
*
|
||||
* The Naming Convention for a RunOptions Config Key,
|
||||
* "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
|
||||
* Such as "ep.cuda.use_arena"
|
||||
* The Config Key cannot be empty
|
||||
* The maximum length of the Config Key is 128
|
||||
*
|
||||
* The string format of a RunOptions Config Value is defined individually for each Config.
|
||||
* The maximum length of the Config Value is 1024
|
||||
*/
|
||||
|
||||
// Key for enabling shrinkages of user listed device memory arenas.
|
||||
// Expects a list of semi-colon separated key value pairs separated by colon in the following format:
|
||||
// "device_0:device_id_0;device_1:device_id_1"
|
||||
// No white-spaces allowed in the provided list string.
|
||||
// Currently, the only supported devices are : "cpu", "gpu" (case sensitive).
|
||||
// If "cpu" is included in the list, DisableCpuMemArena() API must not be called (i.e.) arena for cpu should be enabled.
|
||||
// Example usage: "cpu:0;gpu:0" (or) "gpu:0"
|
||||
// By default, the value for this key is empty (i.e.) no memory arenas are shrunk
|
||||
static const char* const kOrtRunOptionsConfigEnableMemoryArenaShrinkage = "memory.enable_memory_arena_shrinkage";
|
||||
@ -0,0 +1,144 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
/*
|
||||
* This file defines SessionOptions Config Keys and format of the Config Values.
|
||||
*
|
||||
* The Naming Convention for a SessionOptions Config Key,
|
||||
* "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
|
||||
* Such as "ep.cuda.use_arena"
|
||||
* The Config Key cannot be empty
|
||||
* The maximum length of the Config Key is 128
|
||||
*
|
||||
* The string format of a SessionOptions Config Value is defined individually for each Config.
|
||||
* The maximum length of the Config Value is 1024
|
||||
*/
|
||||
|
||||
// Key for disable PrePacking,
|
||||
// If the config value is set to "1" then the prepacking is disabled, otherwise prepacking is enabled (default value)
|
||||
static const char* const kOrtSessionOptionsConfigDisablePrepacking = "session.disable_prepacking";
|
||||
|
||||
// A value of "1" means allocators registered in the env will be used. "0" means the allocators created in the session
|
||||
// will be used. Use this to override the usage of env allocators on a per session level.
|
||||
static const char* const kOrtSessionOptionsConfigUseEnvAllocators = "session.use_env_allocators";
|
||||
|
||||
// Set to 'ORT' (case sensitive) to load an ORT format model.
|
||||
// If unset, model type will default to ONNX unless inferred from filename ('.ort' == ORT format) or bytes to be ORT
|
||||
static const char* const kOrtSessionOptionsConfigLoadModelFormat = "session.load_model_format";
|
||||
|
||||
// Set to 'ORT' (case sensitive) to save optimized model in ORT format when SessionOptions.optimized_model_path is set.
|
||||
// If unset, format will default to ONNX unless optimized_model_filepath ends in '.ort'.
|
||||
static const char* const kOrtSessionOptionsConfigSaveModelFormat = "session.save_model_format";
|
||||
|
||||
// If a value is "1", flush-to-zero and denormal-as-zero are applied. The default is "0".
|
||||
// When multiple sessions are created, a main thread doesn't override changes from succeeding session options,
|
||||
// but threads in session thread pools follow option changes.
|
||||
// When ORT runs with OpenMP, the same rule is applied, i.e. the first session option to flush-to-zero and
|
||||
// denormal-as-zero is only applied to global OpenMP thread pool, which doesn't support per-session thread pool.
|
||||
// Note that an alternative way not using this option at runtime is to train and export a model without denormals
|
||||
// and that's recommended because turning this option on may hurt model accuracy.
|
||||
static const char* const kOrtSessionOptionsConfigSetDenormalAsZero = "session.set_denormal_as_zero";
|
||||
|
||||
// It controls to run quantization model in QDQ (QuantizelinearDeQuantizelinear) format or not.
|
||||
// "0": enable. ORT does fusion logic for QDQ format.
|
||||
// "1": disable. ORT doesn't do fusion logic for QDQ format.
|
||||
// Its default value is "0"
|
||||
static const char* const kOrtSessionOptionsDisableQuantQDQ = "session.disable_quant_qdq";
|
||||
|
||||
// If set to "1", enables the removal of QuantizeLinear/DequantizeLinear node pairs once all QDQ handling has been
|
||||
// completed. e.g. If after all QDQ handling has completed and we have -> FloatOp -> Q -> DQ -> FloatOp -> the
|
||||
// Q -> DQ could potentially be removed. This will provide a performance benefit by avoiding going from float to
|
||||
// 8-bit and back to float, but could impact accuracy. The impact on accuracy will be model specific and depend on
|
||||
// other factors like whether the model was created using Quantization Aware Training or Post Training Quantization.
|
||||
// As such, it's best to test to determine if enabling this works well for your scenario.
|
||||
// The default value is "0"
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsEnableQuantQDQCleanup = "session.enable_quant_qdq_cleanup";
|
||||
|
||||
// Enable or disable gelu approximation in graph optimization. "0": disable; "1": enable. The default is "0".
|
||||
// GeluApproximation has side effects which may change the inference results. It is disabled by default due to this.
|
||||
static const char* const kOrtSessionOptionsEnableGeluApproximation = "optimization.enable_gelu_approximation";
|
||||
|
||||
// Enable or disable using device allocator for allocating initialized tensor memory. "1": enable; "0": disable. The default is "0".
|
||||
// Using device allocators means the memory allocation is made using malloc/new.
|
||||
static const char* const kOrtSessionOptionsUseDeviceAllocatorForInitializers = "session.use_device_allocator_for_initializers";
|
||||
|
||||
// Configure whether to allow the inter_op/intra_op threads spinning a number of times before blocking
|
||||
// "0": thread will block if found no job to run
|
||||
// "1": default, thread will spin a number of times before blocking
|
||||
static const char* const kOrtSessionOptionsConfigAllowInterOpSpinning = "session.inter_op.allow_spinning";
|
||||
static const char* const kOrtSessionOptionsConfigAllowIntraOpSpinning = "session.intra_op.allow_spinning";
|
||||
|
||||
// Key for using model bytes directly for ORT format
|
||||
// If a session is created using an input byte array contains the ORT format model data,
|
||||
// By default we will copy the model bytes at the time of session creation to ensure the model bytes
|
||||
// buffer is valid.
|
||||
// Setting this option to "1" will disable copy the model bytes, and use the model bytes directly. The caller
|
||||
// has to guarantee that the model bytes are valid until the ORT session using the model bytes is destroyed.
|
||||
static const char* const kOrtSessionOptionsConfigUseORTModelBytesDirectly = "session.use_ort_model_bytes_directly";
|
||||
|
||||
/// <summary>
|
||||
/// Key for using the ORT format model flatbuffer bytes directly for initializers.
|
||||
/// This avoids copying the bytes and reduces peak memory usage during model loading and initialization.
|
||||
/// Requires `session.use_ort_model_bytes_directly` to be true.
|
||||
/// If set, the flatbuffer bytes provided when creating the InferenceSession MUST remain valid for the entire
|
||||
/// duration of the InferenceSession.
|
||||
/// </summary>
|
||||
static const char* const kOrtSessionOptionsConfigUseORTModelBytesForInitializers =
|
||||
"session.use_ort_model_bytes_for_initializers";
|
||||
|
||||
// This should only be specified when exporting an ORT format model for use on a different platform.
|
||||
// If the ORT format model will be used on ARM platforms set to "1". For other platforms set to "0"
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsQDQIsInt8Allowed = "session.qdqisint8allowed";
|
||||
|
||||
// x64 SSE4.1/AVX2/AVX512(with no VNNI) has overflow problem with quantizied matrix multiplication with U8S8.
|
||||
// To avoid this we need to use slower U8U8 matrix multiplication instead. This option, if
|
||||
// turned on, use slower U8U8 matrix multiplications. Only effective with AVX2 or AVX512
|
||||
// platforms.
|
||||
static const char* const kOrtSessionOptionsAvx2PrecisionMode = "session.x64quantprecision";
|
||||
|
||||
// Specifies how minimal build graph optimizations are handled in a full build.
|
||||
// These optimizations are at the extended level or higher.
|
||||
// Possible values and their effects are:
|
||||
// "save": Save runtime optimizations when saving an ORT format model.
|
||||
// "apply": Only apply optimizations available in a minimal build.
|
||||
// ""/<unspecified>: Apply optimizations available in a full build.
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsConfigMinimalBuildOptimizations =
|
||||
"optimization.minimal_build_optimizations";
|
||||
|
||||
// Note: The options specific to an EP should be specified prior to appending that EP to the session options object in
|
||||
// order for them to take effect.
|
||||
|
||||
// Specifies a list of stop op types. Nodes of a type in the stop op types and nodes downstream from them will not be
|
||||
// run by the NNAPI EP.
|
||||
// The value should be a ","-delimited list of op types. For example, "Add,Sub".
|
||||
// If not specified, the default set of stop ops is used. To specify an empty stop ops types list and disable stop op
|
||||
// exclusion, set the value to "".
|
||||
static const char* const kOrtSessionOptionsConfigNnapiEpPartitioningStopOps = "ep.nnapi.partitioning_stop_ops";
|
||||
|
||||
// Enabling dynamic block-sizing for multithreading.
|
||||
// With a positive value, thread pool will split a task of N iterations to blocks of size starting from:
|
||||
// N / (num_of_threads * dynamic_block_base)
|
||||
// As execution progresses, the size will decrease according to the diminishing residual of N,
|
||||
// meaning the task will be distributed in smaller granularity for better parallelism.
|
||||
// For some models, it helps to reduce the variance of E2E inference latency and boost performance.
|
||||
// The feature will not function by default, specify any positive integer, e.g. "4", to enable it.
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsConfigDynamicBlockBase = "session.dynamic_block_base";
|
||||
|
||||
// This option allows to decrease CPU usage between infrequent
|
||||
// requests and forces any TP threads spinning stop immediately when the last of
|
||||
// concurrent Run() call returns.
|
||||
// Spinning is restarted on the next Run() call.
|
||||
// Applies only to internal thread-pools
|
||||
static const char* const kOrtSessionOptionsConfigForceSpinningStop = "session.force_spinning_stop";
|
||||
|
||||
// "1": all inconsistencies encountered during shape and type inference
|
||||
// will result in failures.
|
||||
// "0": in some cases warnings will be logged but processing will continue. The default.
|
||||
// May be useful to expose bugs in models.
|
||||
static const char* const kOrtSessionOptionsConfigStrictShapeTypeInference = "session.strict_shape_type_inference";
|
||||
@ -0,0 +1,18 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
namespace onnxruntime {
|
||||
|
||||
// data types for execution provider options
|
||||
|
||||
using ProviderOptions = std::unordered_map<std::string, std::string>;
|
||||
using ProviderOptionsVector = std::vector<ProviderOptions>;
|
||||
using ProviderOptionsMap = std::unordered_map<std::string, ProviderOptions>;
|
||||
|
||||
} // namespace onnxruntime
|
||||
@ -0,0 +1,14 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#include "onnxruntime_c_api.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_Tensorrt, _In_ OrtSessionOptions* options, int device_id);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading…
Reference in New Issue