Pipecom conversion to BAS only (MAJOR UPDATE!)

[Richard Frost]

This is amazing, and despite studying the code for an hour I cannot
figure out how it works.  I need a new brain.

Some variable/offset names are exciting - Process?  Thread?  Can you
do multithreading?  If so, please write a demo - calculating and
plotting Mandelbrots would be something most could understand.

[dbox]

Thanks for this great utility.  Wanted to let you know about an issue I found on Linux.  If the command I send to pipecom is too long, it causes errors indicating that the command text is being truncated.  If I use an arbitrarily large fixed length string instead it seems to work properly.

Just changed the following line:

Code: QB64: [Select]

1. stream = popen(cmd + " 2>pipestderr", "r")

to this:

Code: QB64: [Select]

1. Dim cmdf as String * 200
2. cmdf = cmd + " 2>pipestderr"
3. stream = popen(cmdf, "r")

[George McGinn]

I pass in a Linux application I am writing to pipecom a series of SQL commands (where cmd is more than 2,600 characters) with no issues.

@dbox - Is this only an issue with output to stderr?

The code that Linux executes is:

Code: QB64: [Select]

1. stream = popen(cmd + " 2>pipestderr" + Chr$(0), "r")

Are you running with the current version? Below is the version I am running that I believe is the most current:

Code: QB64: [Select]

1. $If PIPECOM = UNDEFINED Then
2.     $Let PIPECOM = TRUE
3.     Function pipecom& (cmd As String, stdout As String, stderr As String)
4.         stdout = "": stderr = ""
5.         $If WIN Then
6.             Type SECURITY_ATTRIBUTES
7.                 As _Unsigned Long nLength
8.                 $If 64BIT Then
9.                     As String * 4 padding
10.                 $End If
11.                 As _Offset lpSecurityDescriptor
12.                 As Long bInheritHandle
13.                 $If 64BIT Then
14.                     As String * 4 padding2
15.                 $End If
16.             End Type
17.  
18.             Type STARTUPINFO
19.                 As Long cb
20.                 $If 64BIT Then
21.                     As String * 4 padding
22.                 $End If
23.                 As _Offset lpReserved, lpDesktop, lpTitle
24.                 As _Unsigned Long dwX, dwY, dwXSize, dwYSize, dwXCountChars, dwYCountChars, dwFillAttribute, dwFlags
25.                 As _Unsigned Integer wShowWindow, cbReserved2
26.                 $If 64BIT Then
27.                     As String * 4 padding2
28.                 $End If
29.                 As _Offset lpReserved2, hStdInput, hStdOutput, hStdError
30.             End Type
31.  
32.             Type PROCESS_INFORMATION
33.                 As _Offset hProcess, hThread
34.                 As _Unsigned Long dwProcessId
35.                 $If 64BIT Then
36.                     As String * 4 padding
37.                 $End If
38.             End Type
39.  
40.             Const STARTF_USESTDHANDLES = &H00000100
41.             Const CREATE_NO_WINDOW = &H8000000
42.  
43.             Const INFINITE = 4294967295
44.             Const WAIT_FAILED = &HFFFFFFFF
45.  
46.             Declare CustomType Library
47.                 Function CreatePipe& (ByVal hReadPipe As _Offset, Byval hWritePipe As _Offset, Byval lpPipeAttributes As _Offset, Byval nSize As _Unsigned Long)
48.                 Function CreateProcess& (ByVal lpApplicationName As _Offset, Byval lpCommandLine As _Offset, Byval lpProcessAttributes As _Offset, Byval lpThreadAttributes As _Offset, Byval bInheritHandles As Long, Byval dwCreationFlags As _Unsigned Long, Byval lpEnvironment As _Offset, Byval lpCurrentDirectory As _Offset, Byval lpStartupInfo As _Offset, Byval lpProcessInformation As _Offset)
49.                 Function GetExitCodeProcess& (ByVal hProcess As _Offset, Byval lpExitCode As _Offset)
50.                 Sub HandleClose Alias "CloseHandle" (ByVal hObject As _Offset)
51.                 Function ReadFile& (ByVal hFile As _Offset, Byval lpBuffer As _Offset, Byval nNumberOfBytesToRead As _Unsigned Long, Byval lpNumberOfBytesRead As _Offset, Byval lpOverlapped As _Offset)
52.                 Function WaitForSingleObject~& (ByVal hHandle As _Offset, Byval dwMilliseconds As _Unsigned Long)
53.             End Declare
54.  
55.             Dim As Long ok: ok = 1
56.             Dim As _Offset hStdOutPipeRead, hStdOutPipeWrite, hStdReadPipeError, hStdOutPipeError
57.             Dim As SECURITY_ATTRIBUTES sa: sa.nLength = Len(sa): sa.lpSecurityDescriptor = 0: sa.bInheritHandle = 1
58.  
59.             If CreatePipe(_Offset(hStdOutPipeRead), _Offset(hStdOutPipeWrite), _Offset(sa), 0) = 0 Then
60.                 pipecom = -1
61.                 Exit Function
62.             End If
63.  
64.             If CreatePipe(_Offset(hStdReadPipeError), _Offset(hStdOutPipeError), _Offset(sa), 0) = 0 Then
65.                 pipecom = -1
66.                 Exit Function
67.             End If
68.  
69.             Dim As STARTUPINFO si
70.             si.cb = Len(si)
71.             si.dwFlags = STARTF_USESTDHANDLES
72.             si.hStdError = hStdOutPipeError
73.             si.hStdOutput = hStdOutPipeWrite
74.             si.hStdInput = 0
75.             Dim As PROCESS_INFORMATION procinfo
76.             Dim As _Offset lpApplicationName
77.             Dim As String lpCommandLine: lpCommandLine = "cmd /c " + cmd + Chr$(0)
78.             Dim As _Offset lpProcessAttributes, lpThreadAttributes
79.             Dim As Long bInheritHandles: bInheritHandles = 1
80.             Dim As _Unsigned Long dwCreationFlags: dwCreationFlags = CREATE_NO_WINDOW
81.             Dim As _Offset lpEnvironment, lpCurrentDirectory
82.             ok = CreateProcess(lpApplicationName, _Offset(lpCommandLine), lpProcessAttributes, lpThreadAttributes, bInheritHandles, dwCreationFlags, lpEnvironment, lpCurrentDirectory, _Offset(si), _Offset(procinfo))
83.  
84.             If ok = 0 Then
85.                 pipecom = -1
86.                 Exit Function
87.             End If
88.  
89.             HandleClose hStdOutPipeWrite
90.             HandleClose hStdOutPipeError
91.  
92.             Dim As String buf: buf = Space$(4096 + 1)
93.             Dim As _Unsigned Long dwRead
94.             While ReadFile(hStdOutPipeRead, _Offset(buf), 4096, _Offset(dwRead), 0) <> 0 And dwRead > 0
95.                 buf = Mid$(buf, 1, dwRead)
96.                 GoSub RemoveChr13
97.                 stdout = stdout + buf
98.                 buf = Space$(4096 + 1)
99.             Wend
100.  
101.             While ReadFile(hStdReadPipeError, _Offset(buf), 4096, _Offset(dwRead), 0) <> 0 And dwRead > 0
102.                 buf = Mid$(buf, 1, dwRead)
103.                 GoSub RemoveChr13
104.                 stderr = stderr + buf
105.                 buf = Space$(4096 + 1)
106.             Wend
107.  
108.             Dim As Long exit_code, ex_stat
109.             If WaitForSingleObject(procinfo.hProcess, INFINITE) <> WAIT_FAILED Then
110.                 If GetExitCodeProcess(procinfo.hProcess, _Offset(exit_code)) Then
111.                     ex_stat = 1
112.                 End If
113.             End If
114.  
115.             HandleClose hStdOutPipeRead
116.             HandleClose hStdReadPipeError
117.             If ex_stat = 1 Then
118.                 pipecom = exit_code
119.             Else
120.                 pipecom = -1
121.             End If
122.  
123.             Exit Function
124.  
125.             RemoveChr13:
126.             Dim As Long j
127.             j = InStr(buf, Chr$(13))
128.             Do While j
129.                 buf = Left$(buf, j - 1) + Mid$(buf, j + 1)
130.                 j = InStr(buf, Chr$(13))
131.             Loop
132.             Return
133.         $Else
134.             Declare CustomType Library
135.                                 Function popen%& (cmd As String, readtype As String)
136.                                 Function feof& (ByVal stream As _Offset)
137.                                 Function fgets$ (str As String, Byval n As Long, Byval stream As _Offset)
138.                                 Function pclose& (ByVal stream As _Offset)
139.             End Declare
140.  
141.             Declare Library
142.                                 Function WEXITSTATUS& (ByVal stat_val As Long)
143.             End Declare
144.  
145.             Dim As _Offset stream
146.  
147.             Dim buffer As String * 4096
148.             If _FileExists("pipestderr") Then
149.                                 Kill "pipestderr"
150.             End If
151.             stream = popen(cmd + " 2>pipestderr" + Chr$(0), "r")
152.             If stream Then
153.                                 While feof(stream) = 0
154.                                         If fgets(buffer, 4096, stream) <> "" And feof(stream) = 0 Then
155.                                                 stdout = stdout + Mid$(buffer, 1, InStr(buffer, Chr$(0)) - 1)
156.                                         End If
157.                                 Wend
158.                                 Dim As Long status, exit_code
159.                                 status = pclose(stream)
160.                                 exit_code = WEXITSTATUS(status)
161.                                 If _FileExists("pipestderr") Then
162.                                         Dim As Integer errfile
163.                                         errfile = FreeFile
164.                                         Open "pipestderr" For Binary As #errfile
165.                                         If LOF(errfile) > 0 Then
166.                                                 stderr = Space$(LOF(errfile))
167.                                                 Get #errfile, , stderr
168.                                         End If
169.                                         Close #errfile
170.                                         Kill "pipestderr"
171.                                 End If
172.                                 pipecom = exit_code
173.                         Else
174.                                 pipecom = -1
175.                         End If
176.         $End If
177.     End Function
178.  
179.     Function pipecom_lite$ (cmd As String)
180.         Dim As Long a
181.         Dim As String stdout, stderr
182.         a = pipecom(cmd, stdout, stderr)
183.         If stderr <> "" Then
184.             pipecom_lite = stderr
185.         Else
186.             pipecom_lite = stdout
187.         End If
188.     End Function
189. $End If
190.  

Thanks for this great utility.  Wanted to let you know about an issue I found on Linux.  If the command I send to pipecom is too long, it causes errors indicating that the command text is being truncated.  If I use an arbitrarily large fixed length string instead it seems to work properly.

Just changed the following line:

Code: QB64: [Select]

1. stream = popen(cmd + " 2>pipestderr", "r")

to this:

Code: QB64: [Select]

1. Dim cmdf as String * 200
2. cmdf = cmd + " 2>pipestderr"
3. stream = popen(cmdf, "r")

[dbox]

@George McGinn - thanks for the reply.

Looks like the version you have is appending a NUL to the end of the string which resolves the issue for me too.  I guess I did not have the latest version.  When I searched for it I was taken here:  https://www.qb64.org/forum/index.php?topic=3641.msg134943#msg134943.  It does not seem to be the same version you have.  It looks like the latest version in @SpriggsySpriggs github matches what you posted.  I'll get any future updates from there.

Thanks again.