Skip to content

Commit 200ec99

Browse files
SQLDBAWithABeardSQLDBAWithABeard
committed
more for Bob
1 parent b89e9ae commit 200ec99

File tree

1 file changed

+239
-0
lines changed

1 file changed

+239
-0
lines changed

‎notebooks/dotNETNotebooks/ForBob/Jobs but it wont work for you.ipynb

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
{
2+
"metadata": {
3+
"kernelspec": {
4+
"name": "powershell",
5+
"display_name": "PowerShell"
6+
},
7+
"language_info": {
8+
"name": "powershell",
9+
"codemirror_mode": "shell",
10+
"mimetype": "text/x-sh",
11+
"file_extension": ".ps1"
12+
}
13+
},
14+
"nbformat_minor": 2,
15+
"nbformat": 4,
16+
"cells": [
17+
{
18+
"cell_type": "markdown",
19+
"source": [
20+
"If we create a script we can all with some output"
21+
],
22+
"metadata": {
23+
"azdata_cell_guid": "4ce862f4-f2b2-43aa-83f9-03af496c35be"
24+
}
25+
},
26+
{
27+
"cell_type": "code",
28+
"source": [
29+
"$script = @\"\n",
30+
"ECHO 'A thing'\n",
31+
"ECHO 'Another thing'\n",
32+
"ECHO 'Yet another thing'\n",
33+
"ECHO 'Maybe this is progress'\n",
34+
"\"@\n",
35+
"\n",
36+
"$cmdScriptBlock = [scriptblock]::Create($script)\n",
37+
" Invoke-Command -ScriptBlock $cmdScriptBlock | Tee-Object -Variable r"
38+
],
39+
"metadata": {
40+
"azdata_cell_guid": "9c728542-c22d-4ab8-83e4-46ef30a216eb"
41+
},
42+
"outputs": [
43+
{
44+
"output_type": "stream",
45+
"name": "stdout",
46+
"text": "A thing\nAnother thing\nYet another thing\nMaybe this is progress\n"
47+
}
48+
],
49+
"execution_count": 1
50+
},
51+
{
52+
"cell_type": "markdown",
53+
"source": [
54+
"and then add it into a framework of your code"
55+
],
56+
"metadata": {
57+
"azdata_cell_guid": "917ae83c-d910-4f93-bc10-35d75db439a3"
58+
}
59+
},
60+
{
61+
"cell_type": "code",
62+
"source": [
63+
"function Set-S3Object {\r\n",
64+
" $script = @\"\r\n",
65+
"ECHO 'A thing'\r\n",
66+
"ECHO 'Another thing'\r\n",
67+
"ECHO 'Yet another thing'\r\n",
68+
"ECHO 'Maybe this is progress'\r\n",
69+
"\"@\r\n",
70+
"\r\n",
71+
" $cmdScriptBlock = [scriptblock]::Create($script)\r\n",
72+
" Invoke-Command -ScriptBlock $cmdScriptBlock | Tee-Object -Variable r ## This output is sent to the variable\r\n",
73+
"}\r\n",
74+
"\r\n",
75+
"function Copy-GlacierToS3 {\r\n",
76+
" Param($file)\r\n",
77+
" $resultObj = [PSCustomObject]@{\r\n",
78+
" LocalFile = $file\r\n",
79+
" Status = \"always\"\r\n",
80+
" }\r\n",
81+
" Set-S3Object\r\n",
82+
" $resultObj.Status = \"Success\"\r\n",
83+
"\r\n",
84+
"\r\n",
85+
" return $resultObj\r\n",
86+
"}\r\n",
87+
"\r\n",
88+
"# DO Migration\r\n",
89+
"$resultList = @()\r\n",
90+
"$filesToMigrate = 'file1','file2','file3'\r\n",
91+
"# foreach, call Copy-GlacierToS3\r\n",
92+
"foreach ($f in $filesToMigrate) {\r\n",
93+
" $res = Copy-GlacierToS3 -File $f\r\n",
94+
" \r\n",
95+
" $resultList += $res\r\n",
96+
" Start-Sleep -Seconds 2\r\n",
97+
"}\r\n",
98+
"\r\n",
99+
"# display results\r\n",
100+
"$resultList # This is where output will occur"
101+
],
102+
"metadata": {
103+
"azdata_cell_guid": "4f1ecb20-9b75-4bae-86b0-375706a82f99"
104+
},
105+
"outputs": [
106+
{
107+
"output_type": "stream",
108+
"name": "stdout",
109+
"text": "A thing\nAnother thing\nYet another thing\nMaybe this is progress\n\nLocalFile Status \n--------- ------ \nfile1 Success\nA thing\nAnother thing\nYet another thing\nMaybe this is progress\nfile2 Success\nA thing\nAnother thing\nYet another thing\nMaybe this is progress\nfile3 Success\n\n\n"
110+
}
111+
],
112+
"execution_count": 2
113+
},
114+
{
115+
"cell_type": "markdown",
116+
"source": [
117+
"So you would have to do something clever with jobs to get the output - like this - Notebooks will give you the output at the end by default but if you run this code in the console you will see what is happening"
118+
],
119+
"metadata": {
120+
"azdata_cell_guid": "c1fa31c3-eeb0-4ad7-b0d9-1eafcfdcb94e"
121+
}
122+
},
123+
{
124+
"cell_type": "code",
125+
"source": [
126+
"$JobName = 'Jobb'\r\n",
127+
"$JobScript = \"Write-Output 'I am a job';Start-Sleep -Seconds 20 ; Write-Output 'I am further in';Start-Sleep -Seconds 20 ; Write-Output 'I have finished'\"\r\n",
128+
"Start-Job -Name $JobName {Write-Output 'I am a job';Start-Sleep -Seconds 20 ; Write-Output 'I am further in';Start-Sleep -Seconds 20 ; Write-Output 'I have finished'}\r\n",
129+
"$job = Get-Job -Name $Jobname\r\n",
130+
"While (($job.State -ne 'Completed') -and ($job.State -ne 'Failed')){\r\n",
131+
" $job = Get-Job -Name $Jobname\r\n",
132+
" $jobresult = Receive-Job -Name $Jobname\r\n",
133+
" $JobResult\r\n",
134+
" Start-Sleep -Seconds 10\r\n",
135+
"}\r\n",
136+
"$jobresult = Receive-Job -Name $Jobname\r\n",
137+
"$JobResult\r\n",
138+
"Remove-Job -Name $JobName -Force\r\n",
139+
""
140+
],
141+
"metadata": {
142+
"azdata_cell_guid": "e98b0ff9-7ed8-486b-8a00-51992794c55b"
143+
},
144+
"outputs": [
145+
{
146+
"name": "stdout",
147+
"text": "\nId Name PSJobTypeName State HasMoreData Location Command \n-- ---- ------------- ----- ----------- -------- ------- \n19 Jobb BackgroundJob Running True localhost Write-Output 'I am a j...\nI am a job\nI am further in\nI have finished\n\n\n",
148+
"output_type": "stream"
149+
}
150+
],
151+
"execution_count": 13
152+
},
153+
{
154+
"cell_type": "markdown",
155+
"source": [
156+
"BUt in this scenario it is really hard to do as the powershell cant read the output of the invoke-comand that is running the S3 upload command in real time as it sends the command off and doesnt get it back until the end"
157+
],
158+
"metadata": {
159+
"azdata_cell_guid": "d6d35795-9297-484c-b654-6d71eaa8e3bf"
160+
}
161+
},
162+
{
163+
"cell_type": "code",
164+
"source": [
165+
"function Set-S3Object {\r\n",
166+
" $script = @\"\r\n",
167+
"ECHO 'A thing'\r\n",
168+
"ECHO 'Another thing'\r\n",
169+
"ECHO 'Yet another thing'\r\n",
170+
"ECHO 'Maybe this is progress'\r\n",
171+
"\"@\r\n",
172+
"\r\n",
173+
" $cmdScriptBlock = [scriptblock]::Create($script)\r\n",
174+
" Invoke-Command -ScriptBlock $cmdScriptBlock | Tee-Object -Variable r ## This output is sent to the variable\r\n",
175+
"}\r\n",
176+
"\r\n",
177+
"function Copy-GlacierToS3 {\r\n",
178+
" Param($file)\r\n",
179+
" $resultObj = [PSCustomObject]@{\r\n",
180+
" LocalFile = $file\r\n",
181+
" Status = \"always\"\r\n",
182+
" }\r\n",
183+
" Set-S3Object\r\n",
184+
" $resultObj.Status = \"Success\"\r\n",
185+
"\r\n",
186+
"\r\n",
187+
" return $resultObj\r\n",
188+
"}\r\n",
189+
"\r\n",
190+
"# Do Migration\r\n",
191+
"$resultList = @()\r\n",
192+
"$filesToMigrate = 'file1', 'file2', 'file3'\r\n",
193+
"# foreach, call Copy-GlacierToS3\r\n",
194+
"foreach ($f in $filesToMigrate) {\r\n",
195+
" $JobName = 'Jobb'\r\n",
196+
" Start-Job -Name $JobName {$res = Copy-GlacierToS3 -File $f }\r\n",
197+
" $job = Get-Job -Name $Jobname\r\n",
198+
" While (($job.State -ne 'Completed') -and ($job.State -ne 'Failed')) {\r\n",
199+
" $job = Get-Job -Name $Jobname\r\n",
200+
" $jobresult = Receive-Job -Name $Jobname\r\n",
201+
" $JobResult\r\n",
202+
" Start-Sleep -Seconds 10\r\n",
203+
" }\r\n",
204+
" $jobresult = Receive-Job -Name $Jobname\r\n",
205+
" $JobResult\r\n",
206+
" Remove-Job -Name $JobName -Force\r\n",
207+
" \r\n",
208+
" $resultList += $res\r\n",
209+
" Start-Sleep -Seconds 2\r\n",
210+
"}\r\n",
211+
"\r\n",
212+
"# display results\r\n",
213+
"$resultList # This is where output will occur"
214+
],
215+
"metadata": {
216+
"azdata_cell_guid": "53ca1ada-60fd-47e6-bfb1-b85d3ab65844"
217+
},
218+
"outputs": [],
219+
"execution_count": null
220+
},
221+
{
222+
"cell_type": "code",
223+
"source": [
224+
"$jobresult"
225+
],
226+
"metadata": {
227+
"azdata_cell_guid": "26ae2c4e-88ca-4523-972a-67980bc1246e"
228+
},
229+
"outputs": [
230+
{
231+
"name": "stdout",
232+
"text": "",
233+
"output_type": "stream"
234+
}
235+
],
236+
"execution_count": 11
237+
}
238+
]
239+
}

0 commit comments

Comments
 (0)