Really unexpected series: What does VB do for us?

Really unexpected Series
Every time I see the wonderful things of the master, I will scream: "Wow, I can't think of it! ". After many such feelings, I found that as long as we use our brains, we can make things unexpected to others. So I want to share these unexpected things with you, hoping to attract more unexpected things.

Really unexpected series: What does VB do for us?
Keywords: VB, underlying, Win32, API, com
Difficulty: Intermediate
Requirements: Be familiar with VB and use the VC debugger to understand Win32 SDK and COM.
VB has always been regarded as having the following advantages and disadvantages: quick startup and high development efficiency; limited capability and low operation efficiency. This is why some software regards VB as the preferred language, and some software certainly does not use VB. Many programmers of VC and Delphi think that development in VB is not free. It makes us easier to do things and makes us less and less room to use. Indeed, there is a conflict between simplicity and functionality. Even if you do not write a line of code, you just need to start a simple action like running an empty form, VB has done a lot of complicated work for us (not just registering the window class, displaying the window, starting the message loop), which is transparent to programmers. While thanking the VB development team for being considerate to our programmers, we can't help but blame why we haven't mentioned these underlying actions in the document. Although these actions may not affect the final program, but we have the right to know, not to mention that these actions sometimes affect our work (I will talk about this impact in VB multithreading later in this series ).
However, you will be disappointed with all the friends who wish to get "undisclosed technical secrets" from this article, because I can know as much as you do, all we can do is to stand out and guess what VB has done in it? So I am not going to bring you back the VB reverse engineering, but I want to clarify some vague concepts through conjecture about the internal work of VB. As the first article in the series, it aims to lay the foundation for the future, so I will point out the necessary knowledge points when necessary. If you do not know, please study relevant books in time to complete the course. For details, see "bibliography".
Finally, I want to declare that all kinds of experiments and inferences I have done in this article are my personal opinions. They cannot guarantee their correctness and do not assume any relevant legal responsibilities.
Okay. Let's get started! First, prepare our weapons. The following tools are mainly used: VB6 Chinese Enterprise Edition + SP5 (nonsense ), there are also spy ++, dependency walk, and Ole Viewer (hereinafter referred to as spy, depend, and oleview). Spy is in the spy directory under common/tools/Vb/of the VB disc, oleview is the oleview in the oletools directory. EXE. Note that there is another ole2vw32. the EXE function is similar, but this document refers to oleview. EXE, and denpend in the unsupprt/depend ). We also need to use VC (the tool mentioned above is available in VC), because we also need to look at the code generated by VB, and friends who are engaged in Advanced VB development must use the VC debugger, better understanding of assembly. Of course, the focus of this article is not here, so it doesn't matter if there is no VC.
Open VB6 to create a standard EXE project. In the "project"-> "Reference" dialog box, there should be four references. The simple point is: 1. Visual Basic for application (VBA) 2. VB runtime Object Library 3. VB object library 4. OLE automation. The first three items are required by any VB project. If you want to avoid them, try to remove the references. What are the functions of these three core types of libraries and the roles they play in the final executable program? This is the first question to be analyzed in this article.
1) Have you figured out the differences between VB, VBA, and vbs?
First, vbs should not be compared with VB and VBA. It is a scripting language written by Microsoft completely from the beginning according to its own ActiveX Scripting specification. Although its syntax structure is very similar to that of VB, however, vbs only relies on automated objects to expand its functions (only later bound). It cannot use implements to implement interfaces and cannot directly use APIs in vbs, without varptr, we can obtain pointer functions, which are unique to VB and VBA in vbs. Of course, this does not mean that vbs is not as powerful as VB or VBA. Windows already provides vbs with powerful functions. We can use vbs as the COM component script, in addition, with the ability to automate objects, vbs can be said to have unlimited capabilities, so there are viruses written in vbs. For programmers, the most important function of vbs is to provide macro functions for their own software, just like the vbs macro function provided in VC. Note that vbs is free, which is different from using VBA in office to provide macro functions. To integrate VBA, you need a low license fee, for the scripting language, see Platform SDK/tools and scripting ages/scripting in msdn. (In the Script Function Article later in this series, I will implement a small software that uses vbs to provide macro functions)
So what is the difference between VB and VBA? Let's start our experiment!
If Office 2000 or later is installed, open oleview and click View typelib under file to view the location in E: /program files/common files/Microsoft shared/VBA/vba6 vbe6.dll Type Library, and then look at the msvbvm60.dll Type Library in the same way, you will find that their type libraries are basically the same, except that vbe6 has a vbeglobal interface and the global object that implements this interface, we can also see this global object in the VBA programming environment (for example, in the word VB Editor) using an Object Browser. It has two methods: Load And Unload, and a userforms attribute, because vba6 uses the MS form 2.0 Form Designer (fm1_dll) to design and use the userform (while in VB6, we can use multiple designers. For example, by using the MS form 2.0 Form Designer, we can use the userform used by VBA in VB ). Similar to the Global Object of VBA, VB also has a global object, which can be known from the VB Object Browser in the vb6.olb Type Library, this Type Library is the VB object library that every project must reference. All VB built-in objects are here. The objects used in the userform of VBA are all in fm1_dll.
In addition to the preceding differences, the biggest difference between VB and VBA is that VBA cannot generate EXE executable files, however, we can guess that in the IDE environment, both VBA and VB need to compile the code into P-code for execution. Later, I will use experiments to prove that this is true, though VB and VBA differ greatly in their implementations.
From the above analysis, we can see that VB and VBA are still quite different. This difference is mainly reflected in the programming environment and object structure, but in essence there is a blood source relationship between them. If you have carefully observed the msvbvm60.dll Type Library, you will find the following parts:
// Generated. IDL file (by the OLE/COM Object viewer)
[
Dllname ("vba6.dll "),
UUID (35bfbda0-2bcc-1069-82d5-00dd010edfaa ),
Helpcontext (0x000f6ec4)
]
Module strings {
[Entry (0x60000000), helpcontext (0x000f665f)]
Short _ stdcall ASC ([in] BSTR string );
[Entry (0x60000001), helpcontext (0x000f6e9f)]
BSTR _ stdcall _ B _str_chr ([in] Long charcode );
...............
}
What? The object method in msvbvm60.dll is defined in vba6.dll ?! Isn't there a vba6.dll In the VB installation directory? Use oleview to look at it. Wow, it is unexpected that it is exactly the same as msvbvm60.dll. What's going on? Let's take a look at the output functions of vba6.dll, msvbvm60.dll, and vbe6.dll. Haha, there are new discoveries. We can find that in the output functions of the three DLL, the vast majority of them are identical functions starting with RTC, for example, 595 rtcmsgbox (what is RTC? Should it be run time component? Control? Code? Does anyone know ?), This indicates that all three DLL files have the same VBA function at runtime.
We use depend to observe vb6.exe. We can find that vb6.exe introduces some of its unique functions starting with EB and tip in vba6.dll, from the names of these functions, we can find that their functions are all ide-related, such as ebshowcode of 79 and tipdeletemodule of 82. Vb6.exe does not introduce any functions starting with RTC (note 1 ). Let's take a look at msvbvm60.dll. Find a compiled file using the msgbox function and observe it with depend, it will introduce the rtcmsgbox function No. 595 output by msvbvm60.dll (note 2 ). In addition, many functions starting with the following line are introduced in msvbvm60.dll, such as _ vbavarabs (note 3 ). In fact, from the three "Notes", we can make some guesses, whether right or wrong, you can first think about it.
If you haven't followed me in the experiment, but just read this article, I guess you should be a little dizzy. If you did these experiments on your own, you should be in a hurry to see the conclusion. Therefore, you must give it a try. It is more important to study the problem than to draw a conclusion.
At least we can come to the conclusion that VB and VBA are sisters of the same sect, but VB is more difficult than VBA. However, my sister is a strong female, but she only needs to make a huge payment. My sister has the ability to give birth, but she is a real woman. Her sister does not have any child, but she is deeply familiar with her husband's path. What about vbs, she is also the daughter of a large family, but she does not have the excellent lineage of VB and VBA sisters. She can only command some automatically obedient objects to work, she is willing to help others, and she doesn't like VBA. I like VB, VBA, and vbs.

2) What does native code do?
Let's take a further step. The Type Library obtained by oleview does not correctly reflect the function entries in the DLL corresponding to each object method, you should have discovered that the entry attribute values of each method in the IDL file obtained by oleview are false such as 0x600000xx. To get the real entry of each method in the Type Library in the DLL, We need to write the segment program by ourselves.
Even in VB, we can easily obtain the type library information, and add the COM Initialization and call code, we can use our own code to implement the callbyname function introduced by VB6 (in "hack com" next to this series, I will talk more about com, as a VB programmer, it is very important to understand COM ). Since the key of this article is not to guide how to use the Type Library in VB, the methods provided below are as simple as possible.
Create a standard EXE project, add a reference to typelib infomation, put a tag named lblinfo in form, and then add the following code:
'Program 1
Private sub form_load ()
Dim otlinfo as typelibinfo
Dim omeminfo as memberinfo
Dim sdllname as string
Dim sordinal as integer

Set otlinfo = TLI. typelibinfofromfile ("msvbvm60.dll ")

Lblinfo = "the math module contains the following methods:" & vbcrlf

For each omeminfo in otlinfo. typeinfos. nameditem ("math"). Members
With omeminfo
. Getdllentry sdllname, vbnullstring, sordinal

Lblinfo = lblinfo &. Name _
& "Defined in" & sdllname ,"_
& "Its number is" & sordinal _
& Vbcrlf
End

Next
End sub
After running, we can know that the ABS method in the math module is defined in vba6.dll and its number is 656. In depend, check the function 656 in vba6.dll, Which is rtcabsvar. Try using vbe6.dll and the result is the same.
Remember the previous note, vb6.exe does not introduce functions starting with RTC. This shows that the VBA method executed in the IDE environment actually calls the methods in the VBA object library through COM (tracking p-code is a nightmare, so I cannot verify what binding method it uses ). Note that rtcmsgbox is introduced in the final executable program mentioned in section 2. As we expect, the final program will call it directly, which is faster than calling COM, but when tracking the final program, I found that the messageboxa API is called only after the rtcmsgbox has been in the long march of 25,000, during which there are multiple com calls to other objects, which is slow! It may be because the modal dialog box is displayed. There are many factors to consider in the multi-process and multi-threaded environment. If you are a programmer crazy about efficiency, you should try to rewrite msgbox using APIs, definitely a lot faster. Let's take a look at note 3. Let's compile the following program into "program 2.exe" using local code (For subsequent experiments, on the compilation tab of project properties, you can set it to "No optimization" and "generate symbolic debugging information" program 2.exe ""):
'Program 2
Private declare sub debugbreak lib "Kernel32 "()
Private sub main ()
Dim I as long, J as long
Dim K
I = & h1234
Debugbreak
K = 1234
J = ABS (k)
J = ABS (I)
Msgbox "SS"
J = varptr (I)
End sub
Observe "program 2.exe" with depend, and we can find that" program 2.exe" does not introduce 595 rtcabsvar while introducing 656 rtcmsgbox as we expected, on the contrary, it introduces _ vbavarabs and _ v1_4abs. If you look at the function name, you will know that one is for variant and the other is for long. This shows that the object ABS in the final generated code can be further processed for VBA functions of different types of optimization, observe that most functions starting with _ VBA are the most basic and commonly used VBA functions. We can say that VBA functions starting with _ VBA are optimized versions of VBA functions starting with RTC, they are basically rewritten by the VB development team. Most of them implement their own functions within the function, while most functions starting with RTC call COM Service objects to complete the work. From the functions starting with _ VBA, we can see that the VB team has made a lot of effort in native code optimization, which is definitely not a blow. It indeed greatly optimizes many scientific computing-related functions. Taking ABS as an example, native code is more than four times faster than P-code. However, not all computing functions have been optimized like this. For example, the RND function does not have an optimization function starting with _ VBA, but directly corresponds to the rtcrandomnext function, although rtcrandomnext has been optimized, it is better to rewrite it as it uses com internally, I don't understand why the VB development team didn't consider writing a corresponding _ vbarnd for it.
Do not think that the above analysis is meaningless, because we can either look at the nature of phenomena or explain phenomena. For example, if we do another experiment and add a class module to your code, you can try to declare a public method with the same name as the internal method (this is a very useful technology, this method will be used in "error handling" after this series). For example, we can declare a public function RND (x) as single, similarly, we can write a msgbox with the same name. However, if you try to declare a public function ABS (x), a prompt box Indicating "missing identifier" appears in VB ", this error occurs when your function name conflicts with the VB keyword. But why is it a function in the math module, ABS is a keyword, RND is not, and VB does not tell you why, but if you carefully read the experiment analysis above, we can guess this is because VB has done a high degree of Optimization on the functions that require further optimization. The VB development team shows their confidence in their optimization technologies to protect their labor results, we are not allowed to rewrite these functions. At the same time, the VB development team admitted that some functions need to be further optimized, so we are allowed to rewrite them. Here I would like to make a great conjecture that all functions that can be rewritten can be optimized, just as any even number greater than 2 can be broken down into two prime factors and the same.
When it comes to optimization, we should also talk about the differences between direct API calls and the use of API-type libraries. We must also talk about the backend optimizer used by VB (same as the optimizer used by VC ), I also want to talk about how to use vtable binding as much as possible ...... (I am going to write another article "optimization" in this series to talk about these issues ).
After reading the local code, let's take a look at P-Code. If you read the principles of p-code in msdn, you will certainly be the first. In all fairness, p-code is really a great technology, and the code size can be reduced by an average of 50%. Let's compile program 2 into P-code, or use depend for observation. We found that it didn't introduce the function starting with _ VBA (didn't use the optimized VBA function ?), But introduced such a thing as callengine (certainly to call the p-code pseudo code interpretation engine ), rtcmsgbox is introduced in the same way as native code (the p-code generated by compilation should be faster than the p-code running in the IDE environment when msgbox is called ).
If you can't wait to run program 2, you will find that it will pop up an application error dialog box, saying that the program has encountered an exception. Don't be afraid. This is because the debugbreak API is called. In fact, this API generates an int 3 interrupt, allowing us to interrupt program execution. If you have installed a debugger such as VC that supports instant debugging, you can click "cancel" in the error dialog box to start the debugger to debug the program. That's how I run the tracing program. If you want to see the program disassembly code generated by VB, you can try it by yourself. We can use the same technology to interrupt program execution in VB or vba ide, for example, we can run the above Code of Program 2 in the word VB Editor, thus interrupting the process of word, and observe the p-code generated by VBA. For example, the P-code generated by VB and VBA in IDE is very different.
Therefore, the program running in IDE is completely different from the program generated at the end. Use spy ++ to check the form you run in IDE. You will find that it is under the main thread of VB, that is to say, the forms you use in IDE are similar to those used in VB ide. The resources you apply for when your program runs in IDE also belong to vb ide. Some programs run in the IDE to let the IDE die (writing pure API multithreading in vb5 won't run in the IDE. It's definitely more robust than VB6 IDE ). Some programs may work normally in the IDE, but cannot work after the EXE is generated. In short, when writing system programs, consider the possible problems caused by such differences.

3) How can I boast about VB compilation technology and how can I scold you.
After reading the highly appraised native code, you may have more confidence in what VB has made and your waist is straight. Yes, as a VB programmer, there is nothing to be shy about. a vb programmer with profound skill should pay more than a common VC programmer, because his productivity is several times that of a VC programmer, the quality of the program is almost the same as that of VC.
Some masters even joked that the built-in objects of VB are written in VB. For example, we can write the form by ourselves. CLS, label. CTL, haha, we really cannot exclude this possibility (although it is impossible to generate vb6.olb directly with VB ). If so, it seems that the VB team is very confident in its own compilation optimization technology.
The compiler is also VC, so we can rest assured of the backend optimization compiler and connection of the vbprogram. They are basically the VC development team, or both VB and VC are the same compiler development team working on the Compilation module. In short, we can boldly say that the second optimization and connection of the program we do with VB uses the same technology as VC, what you don't have (purely sophistry ).
Moreover, no compiler is faster than the VB compiler, because in IDE, VB is an interpreted language. This is the key to the high development efficiency of VB, and it is almost impossible to compile the process. During request compilation, the background compilation technology is even more outstanding! Think about how much time programmers in other languages spend waiting for code compilation and re-connection!
Don't be too happy, because the ultimate goal is to generate executable files. In VB, there is no function of multipart compilation and incremental join. When VB generates an executable program, it always compiles all modules and completely reconnects them, in other compilation languages, we can only compile the recently modified files (multipart compilation), and add the newly generated code to the end of the executable program, and mark the original code as void (incremental join, the final executable program will become larger and larger, but the connection time is greatly shortened ). When doing the experiment, we will find that the time spent every time the executable file is generated in VB is the same. I don't know why the VB development team does not provide the multipart compilation and incremental join functions. Maybe the VB development team thinks that generating executable files is not a common task in VB. But in fact, this reason cannot be said, because as mentioned above, the running program in IDE is very different from the final program. For example, we need to compile executable files frequently to really profile it, another example is that we cannot debug a multi-threaded program in VB ide. In such cases, the executable file must be re-generated after each modification, we have wasted a lot of time compiling compiled code and connecting connected programs. I guess this is because global optimization is performed when VB generates executable programs, so we have to re-compile all the connections. However, a new function is provided so that we can generate debugging versions that can be compiled in blocks without global optimization. It should not be difficult for the VB development team! (I have a work und that is still being tested)
Let's take a look at the vbaexe6.lib in the VB6 installation directory. How can it be bigger than 1 kb? I guess there should be no code in it. Most of them are function addresses like vtable jump tables or some global constants, I don't know. However, it should at least explain that VB can use static join libraries. Why not provide this function to us and give us more options.
Let's take an experiment and look at a standard EXE project. There is only one standard module in it. There is only one sub main in the module, and there is nothing in the sub main, and it is generated into an EXE file. Look, there are more than 16 K shards. If you have time to track the program that does nothing, you will know that it has to do a lot of things, initialize the ERR and app objects, prepare com calls, and prepare the VB and VBA object libraries, I even made preparations for using ActiveX control. You must use the control in the VB Object Library. Look for a few more EXE projects. There are a lot of things that are the same, they are made by a model, and you are not free to choose a model. The same is true for ActiveX projects, both of which are dual interfaces. the ActiveX control you make must be hidden behind an Extender object. Yes, there are many things in VB That You Have No Choice. If you need this kind of freedom, either not using VB, or you have to adopt some unofficial and eccentric skills that are not made public (one of the most important purposes of this series of articles, is to introduce such unofficial Tips ).
As mentioned at the beginning of this article, VB makes it easier for us to do things, and it also makes us lose a lot of freedom. The formula for generating the final code is also adopted. Of course, we should look at this issue in a comprehensive manner, just as the items produced on the production line are not necessarily more refined than those produced by hand, and the poultry in the group are not as delicious as the wild flavor. If you need a refined wild flavor, this means more labor and higher costs, which is contrary to VB's goal of making it easier and cheaper.

4) VB programmers must also have the hack spirit.
The title at the end of this article is a serious digress, but I want to set a hack-filled tone for this series of articles. What is the hack spirit? There is no accurate definition. My understanding is: hack spirit = I always want to explore curiosity in the unknown field + I always want to know why I want to study + I want to come up with my own innovative spirit + I want to solve problems with patience and perseverance. VB programmers also need this spirit.

Finally, we all know that the VB development team has arrived at the. NET express train. We can't say that VB6 will no longer have a new version of VB. Microsoft has used. Net to draw a new circle for us. VB. NET is a new product in this new circle. In the circle, we can fly higher, but the sky outside the circle is bigger, so I am still happy to stand outside the circle and pray for the birth of the real vb7, amen.