forked from copperwater/xNetHack
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Cross-compiling
510 lines (420 loc) · 25.7 KB
/
Cross-compiling
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
Cross-compiling NetHack 3.7 Last edit: December 1, 2019
The NetHack 3.7 build process differs from the build process of previous
versions in some important ways that make it possible to use a cross-compiler
running on one platform (the "host" platform of the build) to produce a binary
NetHack package that can execute on an entirely different platform.
Part A Contents:
A1. Why cross-compile?
A2. Building NetHack 3.6 (before)
A3. Building NetHack 3.7 (going forward)
A4. How was the build procedure reduced to 5 steps?
A5. How can I help with the cross-compiling initiative?
Part B Contents:
B1. Two sets of compiles and procedures
B2. What needs to be built and executed on the HOST?
B3. What needs to be built for the TARGET?
B4. Case sample: msdos
--------------------------------------------------------------------------------
Part A - Cross-compiling NetHack
--------------------------------------------------------------------------------
+--------------------------+
| A1. Why cross-compile? |
+--------------------------+
By using cross-compilers on host platforms with fast processors, plenty of RAM
and storage resources, and an available cross-compiler, it may be possible to
keep or resurrect a working version of NetHack on platforms that are now too
constrained to carry out the build process natively on the platform any more.
Some of the constraints in carrying out a native build on the desired target
may include, but not necessarily be limited to, any of the following:
o Access: Somebody with a working knowledge of the NetHack build process may
not have the desired target build platform available to them. Conversly,
somebody with a keen knowledge of the target platform, and access to it,
may not be all that familiar with the NetHack build process.
o Resources: Address space limitations, insufficient RAM, low amounts of disk
storage, slow processor performance, may impede the ability to execute the
compile process on the target platform.
o Compilers: Some of the native compilers on historical platforms may only
support the particular dialect of C that was popular when the platform and
compiler were in their prime.
Another useful potential result of cross-compiling, is that it paves the way
for carrying out test and production builds of NetHack for multiple target
platforms through automated steps carried out on the host platform(s).
+-------------------------------------+
| A2. Building NetHack 3.6 (before) |
+-------------------------------------+
Very generally, the build of NetHack in past versions required the following
steps to be carried out:
1. Compile and link util/makedefs.
2. Run makedefs repeatedly with different command line options to produce
several output files that are required for:
(a) additional build steps to follow, including some header
files: pm.h, onames.h, date.h.
(b) creation of files, containing information required by,
or about the game during its execution, that are stored in a
portable, platform-independent way, that need to be inserted
into the game package.
(c) creation of files containing information required by, or about
the game during its execution, that are stored in an architecture
and/or platform and/or operating system dependent way, that need
to be inserted into the game package (the quest text format is
one example).
3. Compile and link the level compiler. This step needs to execute
work-alike tools to lex and yacc, or needs to build pre-built lex and
yacc output (.c, .h files) that are provided in the sys/share part of
the NetHack source code tree.
4. Execute the level compiler to read dat/*.des files and create
a set of binary output files that are architecture and/or operating
system dependent on the build platform, for use by the game during
its execution.
5. Compile and link the dungeon compiler. Like the level compiler, this
step needs to execute work-alike tools to lex and yacc, or needs to
build pre-built lex and yacc output (.c, .h files) that are provided
in the sys/share part of the NetHack source code tree.
6. Execute the dungeon compiler to read dat/dungeon.def and create
a set of binary output files that are architecture and/or operating
system dependent on the build platform, for use by the game during
its execution.
7. Compile and link several less critical utilities such as uudecode,
tile-generation utilities, and so forth, all of which need to execute
on the build platform during the build process to produce output files
for use during the game, that are reasonably portable (not architecture
and/or operating system dependent; only the output of the utilities
becomes part of the game package, not the executable utilities
themselves.
8. Compile and link the game itself.
9. Package the game and its required files including the output from
previous steps 2b, 2c, 4, 6, 7 and 8 above.
Steps 1, 2a, 2b, 7, and 9 above are not impediments to cross-compiling NetHack.
Steps 2c, 3, 4, 5, 6 and 8 above are impediments to cross-compiling NetHack.
That's because the files that those steps produce are very much tied to the
platform where the build of NetHack is being carried out. Variations between
platforms (such as 32-bit vs 64-bit, integer sizes, pointer sizes, processor
byte order, data alignment requirements, struct padding and the way bitfields
are stored) impact the portability of those data files between different
platforms and operating systems. If all those things happen to match, the files
might, just might, be usable across platforms, but the chances are against it,
and that certainly cannot be counted on.
+------------------------------------------+
| A3. Building NetHack 3.7 (going forward) |
+------------------------------------------+
Again, very generally, the build of NetHack in 3.7 requires the following
steps to be carried out:
1. Compile and link util/makedefs.
2. Run makedefs repeatedly with different command line options to produce
several output files that are required for:
(a) additional build steps to follow, including some header
files: pm.h, onames.h, date.h.
(b) creation of files, containing information required by,
or about the game during its execution, that are stored in a
portable, platform-independent way, that need to be inserted
into the game package.
3. Compile and link several less critical utilities such as uudecode,
tile-generation utilities, and so forth, all of which need to execute
on the build platform during the build process to produce output files
for use during the game, that are reasonably portable (not architecture
and/or operating system dependent; only the output of the utilities
becomes part of the game package, not the executable utilities
themselves.
4. Compile and link the game components for the TARGET; that includes
NetHack itself, Lua, and any optional regular-expression or window port
libraries that you plan to link into the NetHack game executable.
5. Package the game and its required files including the output from
previous steps 2b, 3 and 4 above.
Step 4 is now the only impediment to cross-compiling NetHack, and is resolved
by executing step 4 using a cross-compiler that runs on the build (host)
platform to produce a resulting binary for the target platform, instead of
executing the native compiler.
+-----------------------------------------------------+
| A4. How was the build procedure reduced to 5 steps? |
+-----------------------------------------------------+
The following are among several design changes planned in NetHack 3.7,
and these specific changes are what altered the build process to make
cross-compiling possible:
o There is no creation of platform-dependent files, such as the quest
text files, by makedefs during the build process. Instead, the quest
text files have been converted to Lua and are inserted into the game
package for processing by the embedded Lua <fixme: interpreter?>
during execution of NetHack.
o There is no build-time level compiler involved. Instead, the level
descriptions have been converted to Lua and are inserted into the game
package for processing by the embeded Lua <fixme: interpreter?>
during execution of NetHack.
o There is no build-time dungeon compiler involved. Instead, the dungeon
description has been converted to Lua and is inserted into the game
package for processing by the embeded Lua <fixme: interpreter?>
during execution of NetHack.
o Some of the build and option information that was formerly produced
during build time by makedefs, and contained information about the
build-platform specifically, is now produced at runtime within the
game under a cross-compiled build. As such, it now produces information
applicable to the target NetHack environment, not the build environment.
+------------------------------------------------------------+
| A5. How can I help with the cross-compiling initiative? |
+------------------------------------------------------------+
o If you have a favourite target platform (let's call it XX-Platform for
example purposes) that you'd like to see NetHack be able to run on, do
some research to find out if a cross-compiler exists that:
- produces output for XX-Platform.
- executes on a platform that you use and love (Linux, Windows,
Mac OS X are some examples of platforms that have cross-compilers
for other targets available)
o Then, make the community, devteam, and others aware that you're starting
a cross-compile of NetHack for XX-Platform. You might need to ask some
"starting out" questions initially, and as you get deeper into it, you
might need to ask some tougher questions.
o Perhaps consider forking from NetHack on GitHub, and do the
cross-compiler work there in your fork. Strive to get it to a point where
it's ready to play-test on XX-Platform, or perhaps even use an emulator
of XX-Platform if one is available. We live in a time where plenty do.
Doing your work on a GitHub fork has the following advantages:
- It will make it really simple to integrate your work back into
the NetHack source tree if that's one of your goals.
- It will make it possible and straightforward to merge upstream
NetHack changes into your work for the XX-Platform cross-compile
so that it stays current with the game as it evolves.
- You may get help from others in the form of suggestions, or
pull-requests, or offers to join the development. Chances are,
you aren't the only person out there that would like to
establish/resurrect/maintain NetHack on XX-Platform.
Have fun!
-----------------------------------------------------------------------------
Part B - Cross-compiling details
-----------------------------------------------------------------------------
Part B Contents:
B1. Two sets of compiles and procedures
B2. What needs to be built and executed on the HOST?
B3. What needs to be built for the TARGET?
B4. Case sample: msdos
+-----------------------------------------+
| B1. Two sets of compiles and procedures |
+-----------------------------------------+
The HOST is the platform/place that you're running the build procedures using
the native compiler/linker, and the cross-compiler/linker that runs on the HOST
to build the game for a TARGET platform.
You have to:
1. Build mandatory utilities on the HOST.
2. Execute the mandatory utilities to generate components that will be used
during step 4 to build the game itself, or that will generate components
that will become part of the TARGET game package.
3. Build optional or enhancing utilities on the HOST, execute those
optional or enhancing utilities on the HOST to generate components that
will become part of the TARGET game package.
4. Execute a supported cross-compiler to compile the rest of the game
components like NetHack, Lua, and any optional libraries.
(the word "supported", in this sense, means a compiler that runs on your
HOST build platform, and generates output executable files for the TARGET
platform.
It should be mentioned that you can execute the cross-compile build approach
even to generate binaries for the same platform as the host, where the HOST
compilerand the TARGET compiler are actually one and the same.
+------------------------------------------------------+
| B2. What needs to be built and executed on the HOST? |
+------------------------------------------------------+
On the HOST, here are the mandatory things that have to be built.
a) Using the HOST native compiler, build HOST native utility makedefs
Compile and link the following with these compiler switches:
-DCROSSCOMPILE and -DCROSSCOMPILE_HOST
from sources: util/makedefs.c, src/mdlib.c, src/monst.c, src/objects.c
b) Execute HOST native makedefs utility, util/makedefs, as follows:
util/makedefs -v
util/makedefs -o
util/makedefs -p
util/makedefs -z
util/makedefs -d
util/makedefs -r
util/makedefs -h
util/makedefs -s
c) Using the HOST native compiler, build these additional utilities if your
target platform requires components that they produce. It is important
to note that all of the required source files need to be compiled to
native obj files for linking into the HOST-side utility. Some of the
source files (src/monst.c, src/objects.c) were likely already compiled
as native HOST-side obj files in order to build the native HOST utility
'makedefs' HOST utility above, and you don't need to compile them again
for use in linking other HOST utilities if the HOST-native obj files
produced from them are still around.
NOTE: A few other source files (src/drawing.c) that need to
be compiled for native HOST utilities, also need to be compiled again
later as TARGET obj files for linking into the TARGET executable. It is
important to keep the compiled HOST-side obj files produced by the HOST
native compiler, and the TARGET-side obj files produced by the
cross-compiler separate and distinct from one another. That can be done
either by naming the differing object files a little differently
(perhaps with a suffix), or by placing the HOST-side obj files and the
TARGET-side obj files into different folders during the build process.
Whatever works best for your cross-compile.
util/dlb
from sources: src/dlb.c, src/dlb_main.c, src/alloc.c,
util/panic.c
purpose: For packaging up many files that are
required components of the TARGET game
into a single nhdat or nhdat370 combined
file
util/uudecode
from sources: sys/share/uudecode.c
purpose: convert some binary files, that are
distributed in the NetHack sources in
uuencoded format, back into their
original binary state
util/tilemap
from sources: win/share/tilemap.c
purpose: produce output file src/tile.c that is
required for building TARGET packages with
tile support
util/tile2bmp
from sources: win/share/tile2bmp.c, win/share/tiletext.c,
win/share/tilemap.c, src/drawing.c,
src/monst.c, src/objects.c
purpose: <TBD>
util/gif2txt
from sources: win/share/gifread.c, win/share/tiletext.c,
win/share/tilemap.c, src/drawing.c,
src/monst.c, src/objects.c,
src/alloc.c, util/panic.c
purpose: <TBD>
util/ppmwrite
from sources: win/share/ppmwrite.c, win/share/tiletext.c,
win/share/tilemap.c, src/drawing.c,
src/monst.c, src/objects.c,
src/alloc.c, util/panic.c
purpose: <TBD>
+--------------------------------------------+
| B3. What needs to be built for the TARGET? |
+--------------------------------------------+
For the TARGET side, here are the mandatory things that have to be built via
the HOST-executed cross-compiler that generates code for the TARGET platform.
Using the cross-compiler, build the following targets:
a) NetHack sources (core is mandatory)
With the cross-compiler and linker for the TARGET platform,
cross-compile and link with these compiler switches:
-DCROSSCOMPILE and -DCROSSCOMPILE_TARGET
core sources (2019): src/allmain.c, src/apply.c, src/artifact.c,
src/attrib.c, src/ball.c, src/bones.c,
src/botl.c, src/cmd.c, src/dbridge.c, src/decl.c,
src/detect.c, src/dig.c, src/display.c, src/do.c,
src/do_name.c, src/do_wear.c, src/dog.c,
src/dogmove.c, src/dokick.c, src/dothrow.c,
src/drawing.c, src/dungeon.c, src/eat.c, src/end.c,
src/engrave.c, src/exper.c, src/explode.c,
src/files.c, src/fountain.c,
src/hack.c, src/hacklib.c, src/insight.c, src/invent.c,
src/isaac64.c, src/light.c, src/lock.c, src/mail.c,
src/makemon.c, src/mapglyph.c, src/mcastu.c,
src/mdlib.c, src/mhitm.c, src/mhitu.c, src/minion.c,
src/mklev.c, src/mkmap.c, src/mkmaze.c, src/mkobj.c,
src/mkroom.c, src/mon.c, src/mondata.c, src/monmove.c,
src/monst.c, src/mplayer.c, src/mthrowu.c, src/muse.c,
src/music.c, src/nhlsel.c, src/nhlua.c, src/nhlobj.c,
src/o_init.c, src/objects.c, src/objnam.c,
src/options.c, src/pager.c, src/pickup.c, src/pline.c,
src/polyself.c, src/potion.c, src/pray.c,
src/priest.c, src/quest.c, src/questpgr.c,
src/random.c, src/read.c, src/rect.c, src/region.c,
src/restore.c, src/rip.c, src/rnd.c, src/role.c,
src/rumors.c, src/save.c, src/sfstruct.c,
src/shk.c, src/shknam.c, src/sit.c, src/sounds.c,
src/sp_lev.c, src/spell.c, src/steal.c, src/steed.c,
src/sys.c, src/teleport.c, src/tile.c,
src/timeout.c, src/topten.c, src/track.c,
src/trap.c, src/u_init.c, src/uhitm.c, src/vault.c,
src/version.c, src/vis_tab.c, src/vision.c,
src/weapon.c, src/were.c, src/wield.c, src/windows.c,
src/wizard.c, src/worm.c, src/worn.c, src/write.c,
src/zap.c, sys/share/cppregex.cpp
tty sources: win/tty/getline.c, win/tty/termcap.c,
win/tty/topl.c, win/tty/wintty.c
plus your platform-specific source files that contain main, typically
*main.c, and unix support in *unix.c, tty support in *tty.c, and other
system support in *sys.c as well as others sources pertaining to your
specific target platform(s).
b) Lua (mandatory in 3.7)
lib/lua-5.3.5/src
from sources: lua.c, lapi.c, lauxlib.c, lbaselib.c, lcode.c,
lcorolib.c, lctype.c, ldblib.c, ldebug.c,
ldo.c, ldump.c, lfunc.c, lgc.c, linit.c,
liolib.c, llex.c, lmathlib.c, lmem.c,
loadlib.c, lobject.c, lopcodes.c,
loslib.c, lparser.c, lstate.c, lstring.c,
lstrlib.c, ltable.c, ltablib.c, ltm.c,
lundump.c, lutf8lib.c, lvm.c, lzio.c,
lbitlib.c
purpose: links into the game executable to interpret
lua level description files, lua dungeon
description files, and a lua quest text file.
d) recover (optional if desired/required; some targets have recover
functionality built into NetHack itself)
c) Additional optional library packages/obj files as required
lib/pdcurses/...
from sources: addch.c, addchstr.c, addstr.c, attr.c, beep.c,
bkgd.c, border.c, clear.c, color.c, delch.c,
deleteln.c, getch.c, getstr.c, getyx.c,
inch.c, inchstr.c, initscr.c, inopts.c,
insch.c, insstr.c, instr.c, kernel.c,
keyname.c, mouse.c, move.c, outopts.c,
overlay.c, pad.c, panel.c, printw.c,
refresh.c, scanw.c, scr_dump.c, scroll.c,
slk.c, termattr.c, touch.c, util.c, window.c,
debug.c, pdcclip.c, pdcdisp.c, pdcgetsc.c,
pdckbd.c, pdcutil.c
purpose: underlying curses platform support for some
target platforms where inclusion of the
NetHack curses window port in win/curses is
desired
+-------------------------+
| B4. Case sample: msdos |
+-------------------------+
Cross-compiler used: Andrew Wu's djgpp cross-compiler
Cross-compiler url: https://github.com/andrewwutw/build-djgpp
Cross-compiler pre-built binary downloads:
https://github.com/andrewwutw/build-djgpp/releases/download/v2.9/
Mac OS X pre-built binary: djgpp-osx-gcc550.tar.bz2 (tested)
Linux pre-built binary : djgpp-linux64-gcc550.tar.bz2 (tested)
mingw pre-built binary : djgpp-mingw-gcc550-standalone.zip (untested)
The msdos cross-compile for NetHack 3.7 uses two phases of compiles:
Phase1 is the host-side prerequisite stuff that needs to be done first.
Phase2 is the cross-compile pieces using the djgpp cross-compiler hosted on
Linux, Mac OS X, or Windows mingw.
First, on the host platform, you need to set up for a native Unix NetHack
build in the usual way. For example, on linux:
cd sys/unix
sh setup.sh hints/linux
cd ../..
make fetch-lua
Now, you could proceed to go ahead and issue
make all
to build a native NetHack at that point if you wish, but it is not needed
for the msdos cross-compile.
Instead, a test shell script has been put together that will next accomplish
each of the following tasks when it is executed. The shell script can be
invoked by:
sh sys/msdos/msdos-cross-compile.sh
but before you do that, please read the paragraphs below.
The shell script is meant to accomplish the following things:
Prep : the script downloads the djgpp cross-compiler for the host
platform into lib/djgpp (it doesn't install anything on the
system, nor does it need to, it just downloads them into the
identified directories), it downloads a copy of the msdos
dos-extender into lib/djgpp/cwsdpmi for later packaging up with
the msdos game, and it downloads pdcurses into lib/pdcurses
for cross-compiling during the TARGET build.
Be certain to ensure the right products are at the url's
identified above *before* you execute the Case sample msdos
cross-compile script. The correct products were at those url's
at the time this was written in Dec 2019, but we don't assume
any responsibility for what is at those url's now or in the
future. You need to check before executing the script.
Phase1 : the script uses the Makefile sys/msdos/Makefile1.cross
to complete the host-side build steps using the native gcc
compiler for the host platform. During phase1 the host obj
files are put in subfolder src/host_o to keep them separated
and distinguishable from the target obj files that will be
built in phase2.
Phase2 : the script uses the Makefile sys/msdos/Makefile2.cross
to complete the target-side build steps using the
cross-compiler that was obtained during the prep step of the
script described above. During phase2 the target obj files
are put in src/msdos_o to keep them separated and
distinguishable from the host obj files
Package: the script then packages up the results that reside in
msdos-binary into a zip file which it places in lib called
nh370dos.zip.