Author Topic: Rewrite Embellish?!  (Read 7114 times)

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 366
  • Karma: +14/-0
Rewrite Embellish?!
« on: April 07, 2026, 06:14:30 am »
Hello,

Would it be possible for someone to teach the rest of us how to write code, and at the same time rewrite Embellish for e.g. Open Watcom instead of Borland C++ v2?

The reborn source code (contain both windows and OS/2) has been available for quite some time now and it would be very unfortunate if it wasn't put to good use.

While changing the apperance of the buttons of Embellish v2.02 for OS/2 (one can change the looks of applications in OS/2 with rc.exe) I saw that it is restricted to 16 color images of a fixed palette that severly limit the ability to improve the visual appearance of the application.

I wouldn't mind if a programming project to rewrite Embellish would improve stability and allow us to work on larger images etc.
« Last Edit: April 08, 2026, 08:45:22 pm by Jan-Erik Lärka »

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #1 on: April 07, 2026, 07:18:06 am »
Interesting, didn't know that Embellish had been open sourced. Had to use wget to download,
Code: [Select]
wget https://downloads.sourceforge.net/project/embellish/EmbSource.zipProject page, https://sourceforge.net/projects/embellish/
Rewriting for OW is probably mostly a matter of rewriting the make files, perhaps some types need updating. Could also do it with the IDE.
Seems the source also supports compiling with VACPP, though what version I don't know.
Updating the source likely takes some skill. Though even rebuilding with the latest toolkit would likely help with stability and using high memory may be an easy upgrade assuming it is pure 32 bit code.

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 366
  • Karma: +14/-0
Re: Rewrite Embellish?!
« Reply #2 on: April 07, 2026, 07:26:23 pm »
Yes, but why stop there?!

This application is a quite complete package many have used
Embellish contain various features, but why not see if emboss/engrave, antialiasing, floodfill algorithm etc. can be tweaked and/or give more insights.

One goal could be to focus on individual features.
Another to perhaps add support for more file formats
Stability and memory.
GUI improvements, some can't grasp how to use it.
Translations, gui and help.
Separate image manipulation from gui to allow other apps acess to functionality.
Icon editor replacement would require additional development though.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #3 on: April 08, 2026, 03:18:47 am »
The problem is shortage of developers, especially good ones. I've tried learning, and have learned quite a bit, but I'm missing something that is needed to grok code like some do.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1515
  • Karma: +40/-0
Re: Rewrite Embellish?!
« Reply #4 on: April 08, 2026, 04:54:39 am »
The problem is shortage of developers, especially good ones. I've tried learning, and have learned quite a bit, but I'm missing something that is needed to grok code like some do.

Dave makes a great point here...you (we all) need to put in a little "elbow grease" - meaning, hard work - to learn this stuff and get up to speed.

Case in point...so I figured: if this thing is VAC/CPP configured already I might as well take a stab at it, right?

LOL  ::)  8)  :o  >:(

Not quite sure that the source package as-is can be considered 'ready to go'. It is a bit of a mess, confusing directory structure, especially when you consider how the various header files are invoked by the expected makefile execution.

Making things more difficult is the fact that I'm at CPP 3.65 level while this seems to have been a 3.08 configuration, so even some libraries need to be replaced.

But OK, you can get through this, again "a little elbow grease" is what it takes. I actually got some of this compiling now, of cours ILINK is having a hissy fit over a bunch of unresolved externals...but time will tell if I can get through this:

Code: [Select]
...
ILINK /NOFREE /PM:PM /NOD /NOE /EXEPACK:1 /PACKCODE /PACKDATA @JView_os2.lnk

IBM(R) Linker for OS/2(R), Version 03.06.PPK1010917.011116ilink
Copyright (C) IBM Corporation 1988, 1998.
Copyright (C) Microsoft Corp. 1988, 1989.
All rights reserved.

Object Modules [.obj]: JOEVIEWMAIN.OBJ           
Run File [JOEVIEWMAIN.*]: EMB.EXE           
Map File [EMB.map]: JView.map           
Libraries [.lib]: cpprmi36 os2386 misc\EmbMisc.lib filetypes\EmbFiles.lib api\Embapi.lib paint\EmbPnt.lib           
Definitions File [nul.def]: JView_os2.def           
Y:\image\JView_os2.def(10) : warning LNK4087: The obsolete keyword HEAPSIZE is ignored

Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "JVW_DestroyMsgQueue" : unresolved external
Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "InitializePressurePad" : unresolved external
Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "JVW_DestroyAccelTable" : unresolved external
...

It IS a good excercise to go through...just not enough time floating around to invest in each of such little expeditions!

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #5 on: April 08, 2026, 04:45:52 pm »
Even with VACPP 3.08, there are a lot of problems. I don't think the developer finished porting it from Borland to VACPP. Things like inconsistent macro WIN95COMPILE sometimes being WIN95_COMPILE and those missing symbols being a lack of either.
Have to get Martin to create a git repository of the source.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1515
  • Karma: +40/-0
Re: Rewrite Embellish?!
« Reply #6 on: April 09, 2026, 09:28:59 pm »
There is a sucker born every day...and that would be me today it would seem b/c I kept on plugging away on this Embellish thing!

Even with VACPP 3.08, there are a lot of problems. I don't think the developer finished porting it from Borland to VACPP. Things like inconsistent macro WIN95COMPILE sometimes being WIN95_COMPILE and those missing symbols being a lack of either....

So little by little I made more progress, however I'm still stuck at that previously reported error:

Code: [Select]
...
Libraries [.lib]: cpprmi36 os2386 misc\EmbMisc.lib filetypes\EmbFiles.lib api\Embapi.lib paint\EmbPnt.lib           
Definitions File [nul.def]: JView_os2.def           
Y:\image\JView_os2.def(10) : warning LNK4087: The obsolete keyword HEAPSIZE is ignored

Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "JVW_DestroyMsgQueue" : unresolved external
Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "InitializePressurePad" : unresolved external
Y:\image\JOEVIEWMAIN.OBJ(JOEVIEWMAIN.c) : error LNK2029: "JVW_DestroyAccelTable" : unresolved external
...

The above is a result of the following statement:

"ILINK /NOFREE /PM:PM /NOD /NOE /EXEPACK:1 /PACKCODE /PACKDATA @JView_os2.lnk"

where the contents of JView_os2.lnk are:

Quote
JOEVIEWMAIN.OBJ
EMB.EXE
JView.map
cpprmi36 os2386 misc\EmbMisc.lib filetypes\EmbFiles.lib api\Embapi.lib paint\EmbPnt.lib
JView_os2.def

So off I went looking at the embapi.lib library b/c all of the above identifiers are listed in the api.h & api.c source files:

API.C

Code: [Select]
/* This functions terminates the previously initialized */
/* message queue.  This is only used under OS/2  */
#pragma argsused
VOID API_IMPEXP JVW_DestroyMsgQueue(J_HMQ hmq)
{
   #if WIN95_COMPILE
   #else
      WinDestroyMsgQueue((HMQ)hmq);
   #endif
   return;
}

API.H
Code: [Select]
VOID API_IMPEXP  JVW_DestroyMsgQueue(J_HMQ);

Heck...compiling just JOEVIEWMAIN.c by itself seems to be fine, no issues, the OBJ file is created, so that's telling me (I think) that the api.c & api.h are correctly referenced.

So I took a stab at having ILIB take a peek at the contents of embapi.lib, in particular I just did a quick LIST command to see what's there:

Quote
...
- External Definitions:
0060:_JVW_DestroyMsgQueue (OFFSET:0x00000d80, SIZE:0x00000051):

- Public Definitions:
     _JVW_DestroyMsgQueue
...

and so as I stare at this the only thing jumping out at me is this: is the calling convention here the problem? i.e., that "_" character prefixed onto the function name???

Alas, that's the limit of what I can decipher...for all I know the provided LIBs have been created with a different calling convention, _System vs _Optlink (that my IBMCPP 3.6.5 is using by default).

The following paragraph from the on-line IBMCPP help system seems to explain the results I'm seeing:
Quote
You cannot call a function using a different calling convention than the one with which it is compiled. For example, if a function is compiled with _System linkage, you cannot later call it specifying _Optlink linkage.

...but is that the real problem??? LOL

...and if YES, how do I correct/account for it? Do I compile my code with the '/Ms' option???

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #7 on: April 09, 2026, 11:39:37 pm »
IIRC, these were the fixes that got me past the symbol errors. Windows code maybe changing the calling convention.
Code: [Select]
diff --git a/image/api/apidll_os2.mak b/image/api/apidll_os2.mak
index eb6f88c..3670b7a 100644
--- a/image/api/apidll_os2.mak
+++ b/image/api/apidll_os2.mak
@@ -36,10 +36,10 @@ DLL_OBJSDEBUG = \ibmcpp\lib\CPPOPA3.OBJ
 ERROR_CHECK = /Wcnd /Wcns /Wcnv /Weff /Wgen /Word /Wpro /Wret /Wuse
 !IFNDEF NO_DEBUG
 LINK = ILINK /NOFREE /NOE /NOD /DE
-cc = icc -DWIN95COMPILE=0 /Gh+ /Ge- /Gd+ /Se /Gl+ /Gm+ /Ss+ /C /W3 /Gn+ /Ti+ $(ERROR_CHECK)
+cc = icc -DWIN95_COMPILE=0 /Gh+ /Ge- /Gd+ /Se /Gl+ /Gm+ /Ss+ /C /W3 /Gn+ /Ti+ $(ERROR_CHECK)
 !ELSE
 LINK = ILINK /NOFREE /NOD /NOE /EXEPACK:1 /PACKCODE /PACKDATA
-cc = icc -DWIN95COMPILE=0 /Gh- /Ge- /Gd+ /Se /Gl+ /Gm+ /Ss+ /C /W3 /Gn+ /Gi+ /O+ /Ol+ /G5
+cc = icc -DWIN95_COMPILE=0 /Gh- /Ge- /Gd+ /Se /Gl+ /Gm+ /Ss+ /C /W3 /Gn+ /Gi+ /O+ /Ol+ /G5
 !ENDIF
 #-------------------------------------------------------------------
 #   This section lists all files to be built by the make.  The

diff --git a/image/api/standard_include.h b/image/api/standard_include.h
index 44f8c1c..2ab6a5a 100644
--- a/image/api/standard_include.h
+++ b/image/api/standard_include.h
@@ -25,12 +25,14 @@
 //You should have received a copy of the GNU General Public License
 //along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#if WIN95_COMPILE
 #define STRICT
 #include <windows.h>
 #include <process.h>
 #include <commctrl.h>
+#endif
+#include <os2def.h>
 #include <stdarg.h>
 #include <string.h>
 #include <stdlib.h>
The  include <os2def.h> was for a different problem that I'll post about later, so unneeded. Also see if you have the same issue.
BTW, it is possible to use a different calling convention, you need the right pragma or such, forget exactly but needed it for OW code to call GCC compiled DLL's, namely in the screensaver. A lot of the time the solution was already in the headers.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #8 on: April 10, 2026, 01:29:44 am »
Actually, I got side tracked. While those fixes are needed, I'm still getting the unresolved symbols.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #9 on: April 10, 2026, 04:51:20 am »
Out of curiosity and due to knowing GCC much better, I tried building jview with GCC. Added the needed libs one by one and observed the missing symbols. Interestingly some had the _ prefix and some didn't, seems to be two calling conventions. Ended up with one missing symbol. GCC wanted a _prefix and the symbol didn't have one.
Decided to start rebuilding the individual libraries, starting with API. After adding an include <os2.h> it almost succeeded, ending here,
Code: [Select]
Object Modules [.obj]: api.obj tested_memory.obj
Run File [api.*]: EMBAPI.DLL
Map File [EMBAPI.map]: ApiDll.map
Libraries [.lib]: CPPOM30.lib rexx.LIB EmbApi.lib os2386.lib ..\misc\EmbMisc.lib ..\filetypes\EmbFiles.lib ..\paint\EmbPnt.lib
Definitions File [nul.def]: ApiDll_os2.def
ILink : fatal error LNK1082: lnkods50.exe : stub file not found
Have no ideas what lnkods50.exe is, perhaps adds the DLL initterm? Anyways, don't seem to have it on my system.
« Last Edit: April 10, 2026, 04:53:17 am by Dave Yeo »

Jochen Schäfer

  • Sr. Member
  • ****
  • Posts: 414
  • Karma: +33/-0
Re: Rewrite Embellish?!
« Reply #10 on: April 10, 2026, 02:43:16 pm »
So, yes. That's an artifact of early Windows APIs. MS used Pascal calling convention (stack layout and no _ name prefix) for system calls and DLL entry points. They did that to save memory among other things, I can't remember.
C calling convention prefix _ to names and is allowed everywhere else.
Of course, this found it's way into OS/2 ;-)

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 427
  • Karma: +30/-0
  • ONU! (OS/2 is NOT Unix!)
Re: Rewrite Embellish?!
« Reply #11 on: April 10, 2026, 05:13:02 pm »
Code: [Select]
Libraries [.lib]: CPPOM30.lib rexx.LIB EmbApi.lib os2386.lib ..\misc\EmbMisc.lib ..\filetypes\EmbFiles.lib ..\paint\EmbPnt.lib
ILink : fatal error LNK1082: lnkods50.exe : stub file not found
Have no ideas what lnkods50.exe is, perhaps adds the DLL initterm? Anyways, don't seem to have it on my system.

CPPOM30.lib is the VAC 3.0 runtime - clearly not needed for gcc. lnkods50.exe is the DOS stub - again, not needed but probably drawn in by CPPOM30.lib.

Quote
LINK = ILINK /NOFREE /NOE /NOD /DE

/NOD is NoDefaultLibrarySearch - it's the reason the VACPP runtime lib has to be specified; it's hard to see why this was used

/NOE is NoExtDictionary - normally, the linker resolves linkage dependencies (if you link to X the linker will recognize that X links to Y). This disables the extended search and is used when you have to override a built-in like _DLL_Init_Term (sp?). I don't know if this is needed here.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5863
  • Karma: +155/-1
Re: Rewrite Embellish?!
« Reply #12 on: April 12, 2026, 05:42:26 am »
I'm compiling this with VAC 3.08, using the supplied makefiles. Just used GCC to see if better error reporting.
My VAC 3.08 install, kindly donated by Arca Noae, doesn't have lnkods50.exe. I know very little about VAC 3.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1515
  • Karma: +40/-0
Re: Rewrite Embellish?!
« Reply #13 on: April 12, 2026, 05:49:33 pm »
I'm compiling this with VAC 3.08, using the supplied makefiles. Just used GCC to see if better error reporting.
My VAC 3.08 install, kindly donated by Arca Noae, doesn't have lnkods50.exe. I know very little about VAC 3.

Damn...

I'm 3.65 here and having been pulling my hair for the last two days I've made no progress. Tried Rich's sugestions, that actually resulted in more errors, so I stopped there.

Having said that...oh boy, I haven't read so much about compiler calling conventions, the history, the differences - as in stack placement order, etc, etc - like EVER!!!  8)  So there is a tiny little win here after all...

Anyways, Dave, if you get this compiling successfully post here please whatever changes you made b/c I'd love to understand what's causing me the problems I'm seeing here - for all I know it's my environment seeing as I have the IBMCPP 3.65, VAC4, the Toolkit and then the RPM Toolkit all installed here. Mind you, I manage all of this through tool specific environment setup CMD files...still, something can alway sneak through!

Thanks!

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 427
  • Karma: +30/-0
  • ONU! (OS/2 is NOT Unix!)
Re: Rewrite Embellish?!
« Reply #14 on: April 12, 2026, 09:08:33 pm »
I'm compiling this with VAC 3.08, using the supplied makefiles. Just used GCC to see if better error reporting.
My VAC 3.08 install, kindly donated by Arca Noae, doesn't have lnkods50.exe. I know very little about VAC 3.

My mistake: it's provided by ILink 5.0. A copy is attached.