Microsoft.Azure.Management.HDInsight.Job 2.0.5

Microsoft Azure HDInsight Job Management Library

The HDInsight job management client.

Requires NuGet 2.5 or higher.

Install-Package Microsoft.Azure.Management.HDInsight.Job -Version 2.0.5
dotnet add package Microsoft.Azure.Management.HDInsight.Job --version 2.0.5

Release Notes

2.0.5

 * Added support for running queries with large size.
 * Fixed network timeout error which occurs intermittently when HDinsight service is loaded.
 
2.0.4

 * Added support for new environments.
   
   The following API has changed
   public AzureStorageAccess(string storageAccountName, string storageAccountKey, string defaultStorageContainer, string storageAccountSuffix = null)
   Users can optionally specify the storage account URI suffix, for example : "core.chinacloudapi.cn"
   
   users can now use the GetJobOutput() and GetJobErrorLogs() APIs for the new environments

2.0.3

 * Added new API WaitForJobCompletion(string jobId, TimeSpan duration, TimeSpan waitInterval) to wait for completion of a job.

2.0.2
 * This package is now generally available.
 
2.0.1-preview

 * Task cancellation exception can come if http request doesn’t get finished before couple of minutes.

 * If empty job id is passed for Kill job operation then ArgumentException is raised instead of making a kill job request call.
 
 * Added Async APIs for submitting Jobs.

2.0.0-preview:

Some parameters of the job submission objects have changed:

 *  HiveJobSubmissionParameters
      1.   The parameter “Arguments” now has type IList<string>.
      2.   The parameter “Files” now has type IList<string>.
      3.   The parameter “Defines” now has type IDictionary<string, string>.
   
 *  MapReduceJobSubmissionParameters
      1.   The parameter “Arguments” now has type IList<string>.
      2.   The parameter “Files” now has type IList<string>.
      3.   The parameter “LibJars” now has type IList<string>.
      4.   The parameter “Defines” now has type IDictionary<string, string>.
      5.   The parameter “CmdEnv” now has type IDictionary<string, string>.
   
 *  MapReduceStreamingJobSubmissionParameters
      1.   The parameter “Arguments” now has type IList<string>.
      2.   The parameter “Defines” now has type IDictionary<string, string>.
      3.   There is a new parameter “Files” with type IDictionary<string, string>.
   
 *  PigJobSubmissionParameters
      1.   The parameter “Arguments” now has type IList<string>.
      2.   The parameter “Files” now has type IList<string>.
   
 *  SqoopJobSubmissionParameters
      1.   The parameter “Files” now has type IList<string>.
      2.   There is a new parameter “LibDir” with type string.
   
 *  You no longer need to provide a StatusDir argument for job submission requests – one is automatically generated for you.

 *  The unused parameter “UserName” has been removed for all job submission objects. The actual username is automatically picked up from BasicAuthenticationCloudCredentials.

 *  The unused parameter “errorLog” has been removed for all job submission objects. Task log collection is not supported in Hadoop 2.0 or greater. See JIRAs HIVE-6063 and HIVE-6671.

The methods “DownloadJobTaskLogsAsync”, “DownloadJobTaskLogs”, “GetJobTaskLogSummary”, and “GetJobTaskLogSummaryAsync” have been removed. Task log collection is not supported in Hadoop 2.0 or greater. See JIRAs HIVE-6063 and HIVE-6671.

The constructor for HDInsightJobManagementClient now supports providing an HTTP client retry policy for Job REST requests as an optional parameter.

 *  Example:
      public HDInsightJobManagementClient(string clusterDnsName, BasicAuthenticationCloudCredentials credentials, RetryPolicy retryPolicy = null)

 *  If a retry policy is not provided, then a default retry policy (HDInsightJobManagementClient.HDInsightRetryPolicy) will be used.

The methods “GetJobOutput”(+Async) and “GetJobErrorLogsAsync”(+Async) now accept an AzureStorageAccess object as input for storage details instead of three string arguments.

 *  Examples:
      Task<Stream> GetJobOutputAsync(string jobId, IStorageAccess storageAccess, CancellationToken cancellationToken);
      Stream GetJobOutput(this IJobOperations operations, string jobId, IStorageAccess storageAccess)
      Task<Stream> GetJobErrorLogsAsync(string jobId, IStorageAccess storageAccess, CancellationToken cancellationToken);
      Stream GetJobErrorLogs(this IJobOperations operations, string jobId, IStorageAccess storageAccess)

 *  Create the AzureStorageAccess object by providing Azure storage account details:
      new AzureStorageAccess(TestUtils.StorageAccountName, TestUtils.StorageAccountKey, TestUtils.DefaultContainer)

Version History

Version Downloads Last updated
2.0.5 (current version) 2,564 12/16/2016
2.0.4 3,024 11/1/2016
2.0.3 3,359 6/7/2016
2.0.2 592 5/4/2016
2.0.1-preview 119 4/20/2016
2.0.0-preview 958 3/15/2016
1.1.0-preview 660 1/22/2016
1.0.7-preview 1,185 10/21/2015
1.0.6-preview 615 9/23/2015
1.0.5-preview 1,174 9/4/2015
1.0.4-preview 114 9/3/2015
1.0.3-preview 126 9/1/2015
1.0.2-preview 127 8/13/2015
1.0.1-preview 1,543 7/31/2015