From julien at typed-hole.org Sat Jan 4 18:38:26 2020 From: julien at typed-hole.org (julienxx) Date: Sat, 4 Jan 2020 18:38:26 +0000 (UTC) Subject: [ANN] Castor, a graphical Gemini browser In-Reply-To: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> Message-ID: <1D72D8387069DE0E.6DE6D0E5-4F3A-46A0-8E24-D908DECF6871@mail.outlook.com> Added gopher and finger support too! On Tue, Dec 24, 2019 at 5:44 PM +0100, "Julien Blanchard" wrote: Hey Geminauts, I just pushed the initial release of Castor, a GTK Gemini browser. There are no binaries yet but if you have a Rust toolchain installed it should be easy to compile. Source is here https://git.sr.ht/~julienxx/castor I would love some feedback on the app if you have ideas, criticism... Have some nice Winter solstice festivities! -- julienxx -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus at kalasarn.se Tue Jan 7 12:26:18 2020 From: magnus at kalasarn.se (Magnus Wild) Date: Tue, 7 Jan 2020 13:26:18 +0100 Subject: [ANN] Castor, a graphical Gemini browser In-Reply-To: <1D72D8387069DE0E.6DE6D0E5-4F3A-46A0-8E24-D908DECF6871@mail.outlook.com> References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> <1D72D8387069DE0E.6DE6D0E5-4F3A-46A0-8E24-D908DECF6871@mail.outlook.com> Message-ID: <20200107122618.j3gnbheczg7oxfkp@yasuna.kalasarn.se> > I just pushed the initial release of Castor, a GTK Gemini browser. There > are no binaries yet but if you have a Rust toolchain installed it should > be easy to compile. Easy enough, yes. Works well on openbsd too. Thank you for sharing! > Added gopher and finger support too! Works quite nicely too! /Magnus From lazar.michael22 at gmail.com Thu Jan 9 06:00:04 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Thu, 9 Jan 2020 01:00:04 -0500 Subject: Script to test gemini servers Message-ID: Hi all, I have been working on a python script that can be pointed to a gemini server, and will automatically send various types of requests to surface errors and other strange server behavior. This was suggested in the mailing list a while ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it. I am not attempting to cover 100% of the gemini specification. Some of the checks aren't even related to the spec, and are strictly informational. Take all results with a grain of salt. For example: - Does your server support IPv6? - What version of TLS do you negotiate? - Is your certificate's CA trusted? - How do you respond to a http:// URL? - How do you respond to a URL with an invalid UTF-8 byte sequence? - How do you respond to a request with a newline but no carriage return? The program can be found here: https://github.com/michael-lazar/jetforce/blob/master/jetforce_diagnostics.py I've included it in the jetforce repo out of convenience, but the script can be downloaded and run independently of jetforce. The only hard dependency is Python 3.7+. The python "pyca/cryptography" library is an optional requirement, if you want to inspect server TLS certificates. Usage: ./jetforce_diagnostics.py host [port] This is very much still considered a work-in-progress. If you have any suggestions or would like to contribute additional tests to run, please feel encouraged to respond here or open an issue or PR in the github repo. To save you all the trouble, I have already run this against all known gemini servers and published the results on my server. This is not an attempt to shame anybody, my own server exposed several failures that I now need to fix. But I think there are some interesting insights to be gleaned about how servers are implemented in practice, and how the gemini spec might be improved: gemini://mozz.us/diagnostics/ Some of the results: - All servers use "\n" line endings on their home page instead of "\r\n" - No server will accept a URL without a scheme specified (per the spec, this should be implied as gemini://) - Only half of servers currently support TLS version 1.3 - mozz From sean at conman.org Thu Jan 9 07:11:32 2020 From: sean at conman.org (Sean Conner) Date: Thu, 9 Jan 2020 02:11:32 -0500 Subject: Script to test gemini servers In-Reply-To: References: Message-ID: <20200109071132.GA19769@brevard.conman.org> It was thus said that the Great Michael Lazar once stated: > Hi all, Hello. > I have been working on a python script that can be pointed to a gemini server, > and will automatically send various types of requests to surface errors and > other strange server behavior. This was suggested in the mailing list a while > ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it. Thanks for doing this. I find the results interesting, and yes, you did find three bugs in GLV-1.12556. > For example: > > - Does your server support IPv6? Mine (GLV-1.12556) does, but my hosting provider doesn't have IPv6 yet (or maybe they do---it's been awhile since I last checked). > - What version of TLS do you negotiate? 1.2---it's a limitation of the library I'm using. > - Is your certificate's CA trusted? No. I'm the one using a custom CA. > - How do you respond to a http:// URL? I reject them. > - How do you respond to a URL with an invalid UTF-8 byte sequence? Here's one of the bugs, and I have no idea how I'll fix this as it's a cross-library concern. Sigh. > - How do you respond to a request with a newline but no carriage return? I respond. I made CRs optional everywhere internally---it makes it easier for me to test. Now, you did make one test labeled "RequestMissingCR" where you did send a LF. Per the spec, I would expect this to return a 59 Bad Request, but since I treat CRs as optional, that's why I passed that one. > Some of the results: > > - All servers use "\n" line endings on their home page instead of "\r\n" > - No server will accept a URL without a scheme specified (per the spec, this > should be implied as gemini://) This was another bug. I've already fixed this, and the one other bug (I allow URLs longer than 1024 characters, which has been fixed for now). Again, thanks for this. -spc From solderpunk at SDF.ORG Thu Jan 9 19:14:06 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Thu, 9 Jan 2020 19:14:06 +0000 Subject: Script to test gemini servers In-Reply-To: <20200109071132.GA19769@brevard.conman.org> References: <20200109071132.GA19769@brevard.conman.org> Message-ID: <20200109191406.GA29609@SDF.ORG> On Thu, Jan 09, 2020 at 02:11:32AM -0500, Sean Conner wrote: > > Again, thanks for this. > Yes, thanks a lot for this, because I'm continuing to do a bad job of pushing this idea to completion! If you're feeling motivated, some things which my rough stab at this tested which you didn't mention are: * What happens if the client sends a totally empty request? * What happens if the client sends a non-empty request which is not a URL of any kind, like "Hello, Gemini!\r\n"? * What happens if the client sends a non-empty request with a valid URL which is longer than 1024 characters? All of these should result in a status code of 59. Cheers, Solderpunk From lazar.michael22 at gmail.com Fri Jan 10 05:08:31 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Fri, 10 Jan 2020 00:08:31 -0500 Subject: Script to test gemini servers In-Reply-To: <20200109071132.GA19769@brevard.conman.org> References: <20200109071132.GA19769@brevard.conman.org> Message-ID: On Thu, Jan 9, 2020 at 2:11 AM Sean Conner wrote: > > It was thus said that the Great Michael Lazar once stated: > > - How do you respond to a request with a newline but no carriage return? > > I respond. I made CRs optional everywhere internally---it makes it easier > for me to test. Now, you did make one test labeled "RequestMissingCR" where > you did send a LF. Per the spec, I would expect this to return a 59 Bad > Request, but since I treat CRs as optional, that's why I passed that one. It's interesting to hear that this was intentional on your part. You're not alone on this, 3/5 servers that I tested didn't care about the CR at all. My own server does require the CR, but I didn't like writing it that way and felt like I had to jump through unnecessary hoops to conform to the specification: data = await self.reader.readuntil(b"\r\n") instead of the more straightforward: data = await self.reader.readline() I would prefer it if CR was optional. Similar to how is already defined as "any non-zero number of consecutive spaces or tabs". The same argument applies here. Content authors can't visually distinguish newline characters in text/gemini files, and I can't tell the difference in my terminal when I'm inspecting a gemini request. = [] - mozz From solderpunk at SDF.ORG Sun Jan 12 11:36:30 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 12 Jan 2020 11:36:30 +0000 Subject: [ANN] Castor, a graphical Gemini browser In-Reply-To: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> Message-ID: <20200112113630.GA7000@SDF.ORG> Thanks for sharing this, and congrats on being the first graphical Gemini client author! Today I finally managed to get Castor compiled (Rust and Debian Stable are not a winning combiation) and took it for a little test drive. On the whole it seems to work quite nicely, but I did notice some oddities with Gopher content, e.g. gopher://republic.circumlunar.space/~katolaz/phlog/20200111_hack_the_break.txt seems to have the top part of the document missing. I have added Castor to the official list of Gemini software at gemini.circumlunar.space, sorry for the delay in doing so! Cheers, Solderpunk On Tue, Dec 24, 2019 at 05:44:31PM +0100, Julien Blanchard wrote: > Hey Geminauts, > > I just pushed the initial release of Castor, a GTK Gemini browser. There are > no binaries yet but if you have a Rust toolchain installed it should be easy > to compile. > > Source is here https://git.sr.ht/~julienxx/castor > > I would love some feedback on the app if you have ideas, criticism... > > Have some nice Winter solstice festivities! > > -- julienxx > From baschdel at disroot.org Sun Jan 12 13:00:13 2020 From: baschdel at disroot.org (baschdel at disroot.org) Date: Sun, 12 Jan 2020 14:00:13 +0100 Subject: [ANN] Castor, a graphical Gemini browser In-Reply-To: <20200107122618.j3gnbheczg7oxfkp@yasuna.kalasarn.se> References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org> <1D72D8387069DE0E.6DE6D0E5-4F3A-46A0-8E24-D908DECF6871@mail.outlook.com> <20200107122618.j3gnbheczg7oxfkp@yasuna.kalasarn.se> Message-ID: <1578834013.2744.1@disroot.org> Finally got around to trying it out. It's good to see a second graphical gemini browser, also you clearly put some more work into the gtk side of things wich looks really good, than I did with "dragonstone". Thank you for sharing this one! -Baschdel Here is a link to dragonstone: https://gitlab.com/baschdel/dragonstone (I stopped working on it beacause of other projects) From solderpunk at SDF.ORG Sun Jan 12 15:31:39 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 12 Jan 2020 15:31:39 +0000 Subject: [ANN] Announcing Molly Brown, a Gemini server in Go Message-ID: <20200112153139.GB7000@SDF.ORG> Happy 2020, Gemininauts! As I mentioned previously and briefly on this list, over at https://tildegit.org/solderpunk/molly-brown I am working on a Gemini server of my own, named (The Unsinkable) Molly Brown. Molly for short is just fine! The server is written in Go and is intended to be a fully featured server, implementing all aspects of the protocol. This is a very deliberate decision to force me to be personally aware of the implementation difficulty involved in everything that gets specced. Molly is also designed to function in a multi-user environment, such as a public access unix server, where individual users have no access to the main server configuration file and hence need some other way to control features such as redirection or certificate requirements. None of this is implemented yet, but I plan to use .molly files, similar to the .htaccess files used by Apache. I'm very open to other ideas, though, if anybody has any. Molly Brown is now powering the site at gemini.circumlunar.space (which previously ran JetForce). Cheers, Solderpunk From solderpunk at SDF.ORG Sun Jan 12 15:43:21 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 12 Jan 2020 15:43:21 +0000 Subject: [ANN] Announcing free Gemini hosting at gemini.circumlunar.space Message-ID: <20200112154321.GC7000@SDF.ORG> If there is anybody on this list who has been itching to start creating Gemini content but isn't able to or interested in running their own server, I have good news for you! I am now offering free Geminispace which you can manage via sftp at gemini.circumlunar.space. You can choose a username and you'll get a good old fashioned tilde URL, like gemini.circumlunar.space/~solderpunk/. You can host anything there you like which doesn't get me in legal trouble and which I judge not to pose any risk of bringing the Gemini project into ill-repute. For now, I am extending this offer only to people subscribed to this list, to keep things easily manageable while Molly Brown is still under development. If you are interested, please email me (not the whole list!) with your desired username and an SSH public key. Cheers, Solderpunk From solderpunk at SDF.ORG Sun Jan 12 18:43:41 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 12 Jan 2020 18:43:41 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> Message-ID: <20200112184341.GA1011@SDF.ORG> Okay, I have started to re-engage with this endless discussion - slowly and, I have to admit, reluctantly. When I think about how many details there are to consider here, how many different options we have to choose among, and how absolutely incredible the power-to-weight ratio is of verbatim fixed-width text with a predefined width (I mean, really, you can: Left align text, center text, even right align text without the client having to even know what those things are!), it's incredibly tempting to echo the "reflowed text be damned!" sentiment recently expressed at mozz.us[1] and spec 40 character fixed text and just move on. But I know that'd be rash. These days I have been worrying less about catering to small-screened mobile devices and thinking more about the ability for Gemini to be self-documenting. Granted, the text/gemini format is pretty darned simple by design, but nevertheless, having really nice and clear instructions for beginners, many of whom may conceivably never have written either web or gopher content before in their life, will be important to the widespread adoption of the protocol. Imagine how much harder it would be to learn HTML if websites couldn't actually contain any code that you could copy and paste! This is exactly where Gemini is today. Yes, okay, you could put a single space at the start of a link line so it would be displayed as-is rather than treated as a link, and that would be mostly fine. But it would interfere with the ability to simply copy and paste a block of example links and have it work as-is, because the spaces at the front of at least some of the lines would get copied as well. Gopher is better than the current Gemini spec in this regard, because you can put gophermap lines in an item type 0 text file no problem and they'll just be displayed as-is. But copying and pasting that gophermap is not guaranteed to go smoothly. With terminal-based applications, the tabs would stand a good chance of being transformed into consecutive spaces, which would actually break them. Let's be better than that! Let's make it possible to display, copy and paste Gemini links inside of Gemini documents, to facilitate teaching and talking about Gemini over Gemini. It seems quite natural that this should be possible. Even if text/gemini were specced at 40 fixed-width characters with no reflow, meeting this goal would require some syntax comparable to
 tags in HTML, to switch off processing of Gemini links.  If
we're going to have that anyway, we may as well have reflowed text be
the default and this 
 syntax can do double duty by also enabling
non-reflowed text for source code, poetry, etc.

I remain as commited as ever to the idea that any text/gemini markup
should have the property that simple clients can just dump it right to
stdout verbatim and that should be very usable.  As I argued
previously, this rules out any kind of syntax where 
 lines are
indicated by some per-line prefix, as that prefix would break
copy-and-pastability for simple clients dumping things to stdout.
This is one more reason not to use the "leading whitespace" system
proposed by Sean[2] (whose detailed spec remains a very nice and
useful precise definition of lots of fuzzy concepts being slung around
here).

So, given this, I am pretty much settled on using an easily
recognisable line to toggle this mode on or off, say:

```

This is the syntax used in the simplified Markdown proposal at
mozz.us[3], and in ratfactor's "Text Junior" format[4], which others
on this list have argued is a good candidate for Gemini syntax.

I have experimented with supporting this syntax, and allowing
reflowing of text not enclosed by ``` lines to an arbitrary
user-specified width in AV-98 (not pushed to tildegit yet, but expect
it soon).  It really is not too difficult to do, so I don't think
complexity of implementation is a good argument against this.

Does anybody know of a programming language where lines consisting only
of three consecutive single quotes happen to occur frequently?

To address briefly some other points raised in this thread:

Somebody suggested that the ``` syntax or similar be used to toggle on
reflowed text, with fixed text being the default.  I have to admit
this feels really backwards to me.  There are many obvious and common
use cases for wanting to embed small fragments of fixed text inside a
document that one would otherwise want flowed.  I can't think of any
case where I'd want the reverse, but of course if people can I'm happy
to hear them.

I am not crazy about the idea of having text/gemini "source code"
consist of extremely long lines of text which are then brutally
wrapped to a given width by a user's terminal.  Some editors might be
smart enough to present that long line wrapped at word boundaries to
the author, but many won't be and this will result in a very ugly
editing environment.  And I'm not aware of any terminal emulator which
wraps long lines at word boundaries, so this will result in an ugly
reading experience for *all* sized screens.

I also really don't like the idea of supporting colour in Gemini
documents.  I see no way to do this with a syntax which would degrade
gracefully when simply dumped to stdout.  This would also open the
possibility of obnoxious pages with gratuitous use of colour.  No,
thanks!

Cheers,
Solderpunk

[1] gemini://mozz.us/journal/2019-12-05.txt
[2] gemini://gemini.conman.org/gRFC/0004
[3] gemini://mozz.us/markdown/
[4] gopher://sdf.org:70/0/users/ratfactor/phlog/2019-04-21-text-junior

From sean at conman.org  Sun Jan 12 19:59:17 2020
From: sean at conman.org (Sean Conner)
Date: Sun, 12 Jan 2020 14:59:17 -0500
Subject: [ANN] Castor, a graphical Gemini browser
In-Reply-To: <20200112113630.GA7000@SDF.ORG>
References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org>
 <20200112113630.GA7000@SDF.ORG>
Message-ID: <20200112195917.GD23700@brevard.conman.org>

It was thus said that the Great solderpunk once stated:
> Today I finally managed to get Castor compiled (Rust and Debian Stable
> are not a winning combiation) and took it for a little test drive.  On
> the whole it seems to work quite nicely, but I did notice some oddities
> with Gopher content, e.g.
> gopher://republic.circumlunar.space/~katolaz/phlog/20200111_hack_the_break.txt
> seems to have the top part of the document missing.

  That link isn't proper---it should be:

gopher://republic.circumlunar.space/0/~katolaz/phlog/20200111_hack_the_break.txt

  -spc (just an FYI)

From solderpunk at SDF.ORG  Sun Jan 12 20:19:26 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Sun, 12 Jan 2020 20:19:26 +0000
Subject: [ANN] Castor, a graphical Gemini browser
In-Reply-To: <20200112195917.GD23700@brevard.conman.org>
References: <0c8b3f61-1ef9-6ac7-63d2-b3a239931101@typed-hole.org>
 <20200112113630.GA7000@SDF.ORG>
 <20200112195917.GD23700@brevard.conman.org>
Message-ID: <20200112201926.GA14350@SDF.ORG>

Ah, good catch.  I just blindly copied and pasted from the address bar of
Castor, which doesn't seem to iclude item types in the URL.  I don't
think that's related to this error, though, but it might well be.

Cheers,
Solderpunk

On Sun, Jan 12, 2020 at 02:59:17PM -0500, Sean Conner wrote:
> It was thus said that the Great solderpunk once stated:
> > Today I finally managed to get Castor compiled (Rust and Debian Stable
> > are not a winning combiation) and took it for a little test drive.  On
> > the whole it seems to work quite nicely, but I did notice some oddities
> > with Gopher content, e.g.
> > gopher://republic.circumlunar.space/~katolaz/phlog/20200111_hack_the_break.txt
> > seems to have the top part of the document missing.
> 
>   That link isn't proper---it should be:
> 
> gopher://republic.circumlunar.space/0/~katolaz/phlog/20200111_hack_the_break.txt
> 
>   -spc (just an FYI)

From sean at conman.org  Mon Jan 13 22:55:04 2020
From: sean at conman.org (Sean Conner)
Date: Mon, 13 Jan 2020 17:55:04 -0500
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200112184341.GA1011@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
Message-ID: <20200113225504.GF23700@brevard.conman.org>

It was thus said that the Great solderpunk once stated:
> 
> I remain as commited as ever to the idea that any text/gemini markup
> should have the property that simple clients can just dump it right to
> stdout verbatim and that should be very usable.  As I argued
> previously, this rules out any kind of syntax where 
 lines are
> indicated by some per-line prefix, as that prefix would break
> copy-and-pastability for simple clients dumping things to stdout.
> This is one more reason not to use the "leading whitespace" system
> proposed by Sean[2] (whose detailed spec remains a very nice and
> useful precise definition of lots of fuzzy concepts being slung around
> here).

  Thanks.  I understand the concern about copy-n-pasting (I see issues all
the time on Reddit when people don't understand to prepend each line with
four spaces when posting source code).

> So, given this, I am pretty much settled on using an easily
> recognisable line to toggle this mode on or off, say:
> 
> ```

  I have questions.  Oh, do I have questions.


```
This text is on the left.
		This text is mostly centered.
				This text is on the right.
```

But now ...

````
Note that there are four ticks.
  What's the expected result?
    Literal text?
      Or reflowed text?
```

```
This is similar to the above,
  but the block ends with four ticks,
    not the expecte three.
````

``` What about trailing text?
  Is that allowd?
    What is the expected result?
```

```
```

(You should be able to handle the above as well).

> I have experimented with supporting this syntax, and allowing
> reflowing of text not enclosed by ``` lines to an arbitrary
> user-specified width in AV-98 (not pushed to tildegit yet, but expect
> it soon).  It really is not too difficult to do, so I don't think
> complexity of implementation is a good argument against this.

  One thing to watch out for are lines that exceed the wrapping length with
no space (or dash if you are ambitious) characters to break on.

> Does anybody know of a programming language where lines consisting only
> of three consecutive single quotes happen to occur frequently?

  I want to say Python.  I don't program in Python, but I know I have seen
that syntax in some lanuage.

  -spc

> [2] gemini://gemini.conman.org/gRFC/0004

From tomasino at lavabit.com  Mon Jan 13 23:08:58 2020
From: tomasino at lavabit.com (James Tomasino)
Date: Mon, 13 Jan 2020 23:08:58 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200113225504.GF23700@brevard.conman.org>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG> <20200113225504.GF23700@brevard.conman.org>
Message-ID: <50420AAA-DCDB-4180-8ACA-A2327B29C45F@lavabit.com>

> Does anybody know of a programming 
> language where lines consisting only
> of three consecutive single quotes happen to 
> occur frequently?

Just a point of clarification: markdown code fences and your examples use three backticks, not single quotes


From sean at conman.org  Mon Jan 13 23:22:40 2020
From: sean at conman.org (Sean Conner)
Date: Mon, 13 Jan 2020 18:22:40 -0500
Subject: [ANN] Announcing Molly Brown, a Gemini server in Go
In-Reply-To: <20200112153139.GB7000@SDF.ORG>
References: <20200112153139.GB7000@SDF.ORG>
Message-ID: <20200113232240.GG23700@brevard.conman.org>

It was thus said that the Great solderpunk once stated:
> Happy 2020, Gemininauts!
> 
> As I mentioned previously and briefly on this list, over at
> https://tildegit.org/solderpunk/molly-brown I am working on a Gemini
> server of my own, named (The Unsinkable) Molly Brown.  Molly for
> short is just fine!
> 
> The server is written in Go and is intended to be a fully featured
> server, implementing all aspects of the protocol.  This is a very
> deliberate decision to force me to be personally aware of the
> implementation difficulty involved in everything that gets specced.

  I was looking over the code for CGI support and I notice we took two very
different approaches to it.  I based my support upon RFC-3875, with some
changes to reflect Gemini (and support running existing web-based CGI
programs by mapping the return status codes), while you pretty much just run
the script.  I would expect at the very least the QUERY_STRING
meta-variable (as RFC-3875 calls it, or "environment variable" as everybody
else calls them) to be set.

  But given that CGI wasn't specced for Gemini, I think both our
implementations are compliant though.

> Molly is also designed to function in a multi-user environment, such as
> a public access unix server, where individual users have no access to
> the main server configuration file and hence need some other way to
> control features such as redirection or certificate requirements.  None
> of this is implemented yet, but I plan to use .molly files, similar to
> the .htaccess files used by Apache.  I'm very open to other ideas,
> though, if anybody has any.

  I previous did this for GLV-1.12556 but removed the feature [1].  Some
issues you will have to deal with:

	* Syntax errors in the config file and how to handle them.
	* Do you cache the configs?  If you do, how do you get new changes?
	  If you don't, you get a potential performance problem.
	* Do you support configs per directory?

  -spc

[1]	Only for authenticating certificates, and it was per-directory. This
	made it hard to support authentication for non-filesystem handlers
	(which I have a lot of) so I moved authentication to the primary
	config file.

From sean at conman.org  Mon Jan 13 23:23:38 2020
From: sean at conman.org (Sean Conner)
Date: Mon, 13 Jan 2020 18:23:38 -0500
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <50420AAA-DCDB-4180-8ACA-A2327B29C45F@lavabit.com>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG> <20200113225504.GF23700@brevard.conman.org>
 <50420AAA-DCDB-4180-8ACA-A2327B29C45F@lavabit.com>
Message-ID: <20200113232338.GH23700@brevard.conman.org>

It was thus said that the Great James Tomasino once stated:
> > Does anybody know of a programming 
> > language where lines consisting only
> > of three consecutive single quotes happen to 
> > occur frequently?
> 
> Just a point of clarification: markdown code fences and your examples use
> three backticks, not single quotes

  Ah!  That's it!  I knew I saw *something* similar to that in Python.

  -spc


From solderpunk at SDF.ORG  Tue Jan 14 18:43:39 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Tue, 14 Jan 2020 18:43:39 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <50420AAA-DCDB-4180-8ACA-A2327B29C45F@lavabit.com>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 <20200113225504.GF23700@brevard.conman.org>
 <50420AAA-DCDB-4180-8ACA-A2327B29C45F@lavabit.com>
Message-ID: <20200114184339.GA10786@SDF.ORG>

On Mon, Jan 13, 2020 at 11:08:58PM +0000, James Tomasino wrote:
> > Does anybody know of a programming 
> > language where lines consisting only
> > of three consecutive single quotes happen to 
> > occur frequently?
> 
> Just a point of clarification: markdown code fences and your examples use three backticks, not single quotes
> 

Whoops, right you are!  I blame years and years of LaTeX for blurring
the distinction in my mind with it's `unusual quotation syntax'.

Cheers,
Solderpunk

From solderpunk at SDF.ORG  Tue Jan 14 19:03:48 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Tue, 14 Jan 2020 19:03:48 +0000
Subject: [ANN] Announcing Molly Brown, a Gemini server in Go
In-Reply-To: <20200113232240.GG23700@brevard.conman.org>
References: <20200112153139.GB7000@SDF.ORG>
 <20200113232240.GG23700@brevard.conman.org>
Message-ID: <20200114190348.GB10786@SDF.ORG>

On Mon, Jan 13, 2020 at 06:22:40PM -0500, Sean Conner wrote:
> 
>   I was looking over the code for CGI support and I notice we took two very
> different approaches to it.  I based my support upon RFC-3875, with some
> changes to reflect Gemini (and support running existing web-based CGI
> programs by mapping the return status codes), while you pretty much just run
> the script.  I would expect at the very least the QUERY_STRING
> meta-variable (as RFC-3875 calls it, or "environment variable" as everybody
> else calls them) to be set.
> 
>   But given that CGI wasn't specced for Gemini, I think both our
> implementations are compliant though.

Good eye!  I actually intended to post something abou this to the list
shortly after implementing that CGI support, but I suppose that must
have been around the time things started getting really busy for me.

The extremely simple approach to "CGI" (or whatever we want to call
dynamic content) in MollyBrown came about mostly because sloum was keen
to start playing with dynamic content, so I did the simplest thing that
could possibly work - run the script, tell it the corresponding URL over
stdin and then close the pipe.  It can parse out any user input after a
& if it needs it.

This decision should not be interpreted as a criticism of your
RFC-3875-derived implementation.  I think it makes good sense for there
to be an option for people to easily convert existing web CGI scripts to
Gemini.

But I do think it would be nice if there was one vaguely standard way
for servers to implement this kind of thing, so that dynamic content
generating code could be more portable.  I think for that I'd probably
prefer something as light as possible, and to explicitly distance Gemini
from many of the ideas baked into RFC-3875, especially that dynamic
content code should have access to the end user's IP address.

I think there's a lot to recommend the way Molly Brown works, especially
if we generalise it just a little to "Gemini CGI apps should endlessly
read single line URLs over THING, until THING is closed, at which point
the app should terminate".  Here THING could be stdin, or a TCP
connection (making a CGI app basically a small self-contained server),
or a unix domain socket.  Simple servers could do what Molly currently
does, just spawn the script, send a single URL over stdin and then close
stdin, giving us the good old fashioned one-process-per-request model of
traditional web CGI.  But more advanced servers could give admins a way
to configure different approaches where the process is persistent, more
like FastCGI.  Or they could round-robin load balance between multiple
servers on a local network.  The actual CGI program would see very
little difference between these scenarios, you'd just give a slightly
different argument to a library function which produced some kind of
iterator over URLs.  This has great power:weight.

I was very happy with this idea until I realised that CGI programs
should also have some way to get access to client certificates, not just
the URL. :(

I haven't returned since then to thinking about how to achieve this.
Maybe a good subject for a new thread...

Cheers,
Solderpunk

From solderpunk at SDF.ORG  Tue Jan 14 19:34:00 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Tue, 14 Jan 2020 19:34:00 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200113225504.GF23700@brevard.conman.org>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 <20200113225504.GF23700@brevard.conman.org>
Message-ID: <20200114193400.GC10786@SDF.ORG>

On Mon, Jan 13, 2020 at 05:55:04PM -0500, Sean Conner wrote:

It seems easiest to me (and my current experimental implementation in
AV-98 does this) to make the spec quite strict in this regard: verbatim
mode is toggled on/off by lines consisting of precisely 3 ticks and
nothing else.  Which means...
 
> ````
> Note that there are four ticks.
>   What's the expected result?
>     Literal text?
>       Or reflowed text?
> ```

The first 5 lines of this will be reflowed, the final line will trigger
literal text below it.

> ```
> This is similar to the above,
>   but the block ends with four ticks,
>     not the expecte three.
> ````

Literal mode is turned on by the first line but never turned off, so
there will be a literal line of four ticks and that end, and whatever
comes lext will be literalised, too!
 
> ``` What about trailing text?
>   Is that allowd?
>     What is the expected result?
> ```

Literal mode won't be turned on until the very end, and the first three
lines will be flowed.

> ```
> ```
> 
> (You should be able to handle the above as well).

Hmm, I think AV-98 will handle that just fine, hang on...yep, no
problems.

>   One thing to watch out for are lines that exceed the wrapping length with
> no space (or dash if you are ambitious) characters to break on.

Ah, that's an annoying edge case.  I guess such lines can just be broken
at exactly the viewport width, possibly with a dash at the end of all
but the final line?
 
>   I want to say Python.  I don't program in Python, but I know I have seen
> that syntax in some lanuage.

I was briefly worried about Python docstrings (before Tomasino pointed
out we aren't using quotes at all!), but PEP 257 recommends always using
double quotes for them anyway.

Cheers,
Solderpunk

From tomasino at lavabit.com  Tue Jan 14 21:02:52 2020
From: tomasino at lavabit.com (James Tomasino)
Date: Tue, 14 Jan 2020 21:02:52 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200114193400.GC10786@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG> <20200113225504.GF23700@brevard.conman.org>
 <20200114193400.GC10786@SDF.ORG>
Message-ID: <1D76A420-2CB1-4FEA-9B0D-66DE63E220BF@lavabit.com>




-------- Original Message --------
From: solderpunk 

> verbatim mode is toggled on/off by lines 
> consisting of precisely 3 ticks and nothing 
> else.  

Again, just clarifying for people who might not be familiar: the code fencing in markdown often allows the initial backtick trio to be followed by a filetype which aids clients in syntax highlighting. They look like so:

```bash
```javascript
```ruby

I don't suggest gemini needs that, but know that people may assume the behavior because it is familiar. It might be easier to declare lines starting with 3 backticks to be the code fence barriers regardless of what comes later. If clients want to develop extra syntax highlighting they could, but mostly it would mean people pasting code they'd use on GitHub or stackoverflow will not be surprised by broken wrapping.


From sean at conman.org  Tue Jan 14 23:49:20 2020
From: sean at conman.org (Sean Conner)
Date: Tue, 14 Jan 2020 18:49:20 -0500
Subject: CGI and CGI like support (was Re: [ANN] Announcing Molly Brown,
 a Gemini server in Go)
In-Reply-To: <20200114190348.GB10786@SDF.ORG>
References: <20200112153139.GB7000@SDF.ORG>
 <20200113232240.GG23700@brevard.conman.org> <20200114190348.GB10786@SDF.ORG>
Message-ID: <20200114234920.GA13720@brevard.conman.org>

It was thus said that the Great solderpunk once stated:
> 
> This decision should not be interpreted as a criticism of your
> RFC-3875-derived implementation.  I think it makes good sense for there
> to be an option for people to easily convert existing web CGI scripts to
> Gemini.
> 
> But I do think it would be nice if there was one vaguely standard way
> for servers to implement this kind of thing, so that dynamic content
> generating code could be more portable.  I think for that I'd probably
> prefer something as light as possible, and to explicitly distance Gemini
> from many of the ideas baked into RFC-3875, especially that dynamic
> content code should have access to the end user's IP address.

  RFC-3875 wasn't that bad to support as there aren't that many
meta-varables (as they are called) to suport, and several are optional
anyway.  The RFC doesn't cover how the meta-varriables are sent to the
script, but under Unixland, it's via envinroment variables.

Here's what I currently do:

	AUTH_TYPE
		Not set unless the client provides a certificate, then this
		gets set to "Certificate".

	CONTENT_LENGTH
		Doesn't apply as there's no way to send a document to a
		Gemini server.

	CONTENT_TYPE
		Doesn't apply.

	GATEWAY_INTERFACE
		Set to "CGI/1.1"

	PATH_INFO
		Per RFC (wording is a bit muddled), and not always set.

	PATH_TRANSLATED
		Per RFC, and not always set.  I will say that these two are
		a bit persnickity to get right.

	QUERY_STRING
		Must be set.  If no query string, set to "".

	REMOTE_ADDR
	REMOTE_HOST
		I take it these are the ones you oject to the most.  But if
		I'm running a Gemini server, I *already* have your IP
		address anyway.  It seems silly to hide it to me, but I
		don't live in Europe so take what I say with a grain of salt
		or two.  I set these (to just the IP address).

	REMOTE_IDENT
		Nobody supports RFC-1413, so I skip this one.

	REMOTE_USER 
		If a client provides a certificate, I set this to the client
		subject common name.

	REQUEST_METHOD
		I set this to "", as Gemini has no concept of a request
		method (but see below).

	SCRIPT_NAME
		Per RFC.  Not hard to set properly.

	SERVER_NAME 
		Hostname of the current server.  If you support multiple
		hosts per Gemini, then I would set this to the host the
		client connected to.

	SERVER_PORT
		Set to port number of server.

	SERVER_PROTOCOL
		Set to "GEMINI".

	SERVER_SOFTWARE
		Set to "GLV-1.12556/1".

  And that's it without further configuration.  As a default, a CGI script
will ONLY get these environment variables (whereas your implementation leaks
the parent environment to the script---might want to check that).  I allow
one to set other environment variables per script (like $PATH or $LANG or
whatever).  If you need HTTP compatibility, I set some HTTP_* and change
REQUEST_METHOD to "GET" and SERVER_PROTOCOL to "HTTP/1.0".  I also have an
option to set some variables that Apache sets as well.

  If the client presents a certificate, I set the following:

	TLS_CIPHER
	TLS_VERSION
	TLS_CLIENT_HASH
	TLS_CLIENT_ISSUER
	TLS_CLIENT_SUBJECT
	TLS_CLIENT_NOT_BEFORE
	TLS_CLIENT_NOT_AFTER
	TLS_CLIENT_REMAIN (time between now and TLS_CLIENT_NOT_AFTER)
	TLS_CLIENT_ISSUER_* (various fields broken down)
	TLS_CLIENT_SUBJECT_* (various fields broken down)

and AUTH_TYPE and REMOTE_USER as mentioned above (if Apache compatibility
requested, the names change but it's largely the same information).  Details
can be seen starting here:

https://github.com/spc476/GLV-1.12556/blob/master/Lua/GLV-1/cgi.lua#L241
	
> I think there's a lot to recommend the way Molly Brown works, especially
> if we generalise it just a little to "Gemini CGI apps should endlessly
> read single line URLs over THING, until THING is closed, at which point
> the app should terminate".  

  Oh, so pretty much a Gemini server sans TLS then.

> Here THING could be stdin, or a TCP
> connection (making a CGI app basically a small self-contained server),
> or a unix domain socket.  Simple servers could do what Molly currently
> does, just spawn the script, send a single URL over stdin and then close
> stdin, giving us the good old fashioned one-process-per-request model of
> traditional web CGI.  But more advanced servers could give admins a way
> to configure different approaches where the process is persistent, more
> like FastCGI.  Or they could round-robin load balance between multiple
> servers on a local network.  The actual CGI program would see very
> little difference between these scenarios, you'd just give a slightly
> different argument to a library function which produced some kind of
> iterator over URLs.  This has great power:weight.
> 
> I was very happy with this idea until I realised that CGI programs
> should also have some way to get access to client certificates, not just
> the URL. :(
> 
> I haven't returned since then to thinking about how to achieve this.

  Perhaps a series of line, like:

	Request: gemini://example.net/foo/bar/script
	TLS_Cipher: ...
	TLS_Version: ...
	TLS_CLient_Hash: ...
	TLS_Client_Issuer: ...

  Ends with EOF (or a blank line or a NUL byte or some way to indicate the end
of this one request).  This is consistent (each line is formatted the same
way) and I think, easy to deal with.

  But whatever you come up with, I would try to avoid calling it CGI, as
that tends to lead to RFC-3875 ...

  -spc

From lazar.michael22 at gmail.com  Wed Jan 15 05:03:28 2020
From: lazar.michael22 at gmail.com (Michael Lazar)
Date: Wed, 15 Jan 2020 00:03:28 -0500
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200112184341.GA1011@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
Message-ID: 

On Sun, Jan 12, 2020 at 1:43 PM solderpunk  wrote:
>
> Okay, I have started to re-engage with this endless discussion -
> slowly and, I have to admit, reluctantly.  When I think about how many
> details there are to consider here, how many different options we have
> to choose among, and how absolutely incredible the power-to-weight
> ratio is of verbatim fixed-width text with a predefined width (I
> mean, really, you can:
>
> Left align text,
>                              center text,
>                                                  even right align text
>
> without the client having to even know what those things are!), it's
> incredibly tempting to echo the "reflowed text be damned!" sentiment
> recently expressed at mozz.us[1] and spec 40 character fixed text and
> just move on.

For what it's worth, I have been trying
out the 40-character width thing for a
while now and I'm really enjoying it! I
actually find it a lot more pleasant to
type vs 80 character lines. I don't know
if it's because my eyes don't need to
jump as far, or because it takes fewer
keystrokes to move my cursor to the
middle of a line... Something about it
just *feels* good to type.

Not to mention, pages like this [1]
display perfectly on my iphone using a
gemini-http proxy server. Regardless of
whether you choose to adopt the ```
mode, you're still going to need to
recommend a line length for authors to
hard wrap their text/gemini files at.
And I suggest that 40 is still worth
considering for this.

> Gopher is better than the current Gemini spec in this regard, because
> you can put gophermap lines in an item type 0 text file no problem and
> they'll just be displayed as-is.  But copying and pasting that
> gophermap is not guaranteed to go smoothly.  With terminal-based
> applications, the tabs would stand a good chance of being transformed
> into consecutive spaces, which would actually break them.  Let's be
> better than that!  Let's make it possible to display, copy and paste
> Gemini links inside of Gemini documents, to facilitate teaching and
> talking about Gemini over Gemini.  It seems quite natural that this
> should be possible.
>
> Even if text/gemini were specced at 40 fixed-width characters with no
> reflow, meeting this goal would require some syntax comparable to
> 
 tags in HTML, to switch off processing of Gemini links.  If
> we're going to have that anyway, we may as well have reflowed text be
> the default and this 
 syntax can do double duty by also enabling
> non-reflowed text for source code, poetry, etc.

Here are some other alternatives that
might be worth considering. I do think
that displaying gemini links is a valid
use-case, but adding a whole new
preformatted text mode only for this
narrow case feels a bit heavy-handed to
me. Granted, I realize there are other
benefits to the preformatted mode that
have already been outlined.

Option 1. Use a no-op link

Pick a URL that by convention doesn't
lead anywhere useful, and then hijack
the (link friendly name) portion to
display your gemini link.

=># =>/about.txt About

"#" is a valid relative URL, right?
Somebody else on this list *cough* sean
might be able to some up with something
better. This would be displayed on
most gemini clients as:

=>/about.txt About

The line would be highlighted as a link
(unless clients choose to handle this
special case), but otherwise it should
work without any changes to the spec.

Option 2. Use text/plain

For the narrow use-case where you want
to show off some examples of gemini
links, stick those links in a separate
text/plain document. Or just serve your
whole page as text/plain. The example
links can't intermingle with real gemini
links in the same document, but is that
really such a big deal?

How you feel about this option likely
depends on which side of the fence you
fall on regarding text/gemini usage.
Should text/gemini be used like HTML is
on the web, with most content being
written as gemini files? Or should it be
more like gopher, where directories are
type text/gemini but many people write
their blog posts and other leaf
documents as text/plain.

Lately I have been leaning more towards
the second interpretation. Take another
example: Instead of writing a python
code snippet inline in a text/gemini
document, what if you instead added a
link to your code snippet and served it
as "text/x-python"? This feels natural
to me given that other media content
like images also can't be displayed
inline.

[1] https://portal.mozz.us/gemini/mozz.us/diagnostics/2020-01-08/notes.gmi

- mozz

From sean at conman.org  Wed Jan 15 05:21:17 2020
From: sean at conman.org (Sean Conner)
Date: Wed, 15 Jan 2020 00:21:17 -0500
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: 
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
Message-ID: <20200115052117.GB13720@brevard.conman.org>

It was thus said that the Great Michael Lazar once stated:
> 
> Option 1. Use a no-op link
> 
> Pick a URL that by convention doesn't
> lead anywhere useful, and then hijack
> the (link friendly name) portion to
> display your gemini link.
> 
> =># =>/about.txt About
> 
> "#" is a valid relative URL, right?
> Somebody else on this list *cough* sean
> might be able to some up with something
> better. 

  Who?  Me?  Wait!  Did you ask for two examples of a pronoun?  

  Anyway, yes '#' is a vaild URL, or at least, it passes through my URL
parser.  And it's a cute work around.

> This would be displayed on
> most gemini clients as:
> 
> =>/about.txt About
> 
> The line would be highlighted as a link
> (unless clients choose to handle this
> special case), but otherwise it should
> work without any changes to the spec.
> 
> Option 2. Use text/plain

  Option 3, Use text/markdown.  RFCs 7763 and 7764.  

  Other than that, I don't really ha much of a horse in this particular
race.

  -spc


From b__m__e at mailfence.com  Wed Jan 15 19:30:50 2020
From: b__m__e at mailfence.com (Brian Evans)
Date: Wed, 15 Jan 2020 20:30:50 +0100 (CET)
Subject: Text reflow woes (or: I want bullets back!)y
Message-ID: <660019441.8880.1579116650232@ichabod.co-bxl>

Solderpunk wrote recently regarding use of color in gemini
documents. 

I would like to put in a vote to the contrary. I know that not
all clients will support color, those that do not may not be
coded to remove any methodology of adding color (current
methods surround vt100/ansi escape sequences). Many
clients will also not be terminal based, making escape
sequences nearly meaningless. Having said all this, my two
cents are that the spec should not address this whatsoever.
It should be on content creators to decide whether adding
said escape sequences is something that are comfortable 
with. Likewise, each client should choose whether this is
a detail they want to handle or not. At present my client
supports color for all protocols that it supports (gopher,
gemini, finger, local files). When color mode is toggled on
any escape sequences in the \033[???m series will be
rendered (but no other escape sequences). It feels weird
to _remove_ gemini from this feature. It feels like it should
not be a part of the spec, just like it is not part of the
gopher spec, and some clients support it and some dont:
content creators beware.


I am running into similar worries with the wrapping and
reflowing conversation. I do not want to add horizontal
scrolling to my client so hard wrapping will be done at
screen width. Reflowing is another matter of course....
I'm starting to feel like it just isnt worth the fuss. Since
_all_ clients seem to hard wrap, just use a long line if you
want flowed content. Done. No extra work required for
clients. It keeps things very very simple. At present gemini
is great to navigate around (what there is of it anyway)
and fulfills the goal of being like gopher, but better. 

Just my two cents...

--?
Sent with https://mailfence.com
Secure and private email

From solderpunk at SDF.ORG  Wed Jan 15 20:21:37 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Wed, 15 Jan 2020 20:21:37 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: 
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
Message-ID: <20200115202137.GA21763@SDF.ORG>

On Wed, Jan 15, 2020 at 12:03:28AM -0500, Michael Lazar wrote:
> 
> Not to mention, pages like this [1]
> display perfectly on my iphone using a
> gemini-http proxy server. Regardless of
> whether you choose to adopt the ```
> mode, you're still going to need to
> recommend a line length for authors to
> hard wrap their text/gemini files at.
> And I suggest that 40 is still worth
> considering for this.

Yes, I definitely intend to include a recommendation that text/gemini
content be hard wrapped at something less than the traditional 70 or 80
columns.  This convention seems to slowly taking hold in the
phlogosphere (driven, delightfully, as much by retro PDA enthusiasts as
smartphone users!).  In writing about this earlier, I idly raised the
prospect of clients for "real computers" displaying text hard wrapped at
~40 characters displaying multiple columns side by side - I actually
bumped into an example of this in the wild the other day (see the end of
gemini://tilde.black/users/brool/stoned.txt) and thought it looked
great!
 
> Here are some other alternatives that
> might be worth considering. I do think
> that displaying gemini links is a valid
> use-case, but adding a whole new
> preformatted text mode only for this
> narrow case feels a bit heavy-handed to
> me. Granted, I realize there are other
> benefits to the preformatted mode that
> have already been outlined.

I agree that the ability to display literal Gemini links inside a
text/gemini page probably doesn't, by itself, justify adding any kind
of markup complexity.  But there are indeed many other benefits, as you
mentioned.  I had kind of been on the fence about whether or not such
complexity was worth adding mostly on the basis of those other benefits
(which mostly relate to supporting mobile devices, which, even though I
do use them, I kind of hate and feel reluctant to cater to).  The
realisation that a verbatim text mode can also do some genuinely useful
work even on a "big screen" where text reflowing never has to be done
has, I think, tipped the scales for me enough to decide that this is
worth it.

After all, the essential complexity cost we'd be paying for these
benefits is quite low.  Remembering that it will be explicitly okay for
simple clients not to reflow text if they don't want to, the most
anybody will be obligated to do is something like the following:

in_verbatim = False
for line in all_the_lines:
    if line == "```":
        in_verbatim = not in_verbatim
    elif line.startswith("=>"):
        handle_a_link(line)
    else:
        print(line)

This won't display the ``` lines and will avoid trying to parse any
links which are supposed to be presented verbatim to the uesr (which,
for the purposes of education, might be syntactically invalid and not
something the client should try to parse anyway).  Nobody can
reasonably call the above difficult or bloated, and it's only three
lines longer than the bare minimum that is currently required:
 
for line in all_the_lines:
    if line.startswith("=>"):
        handle_a_link(line)
    else:
        print(line)

(as an aside, the times recently I've talked about "simple clients
catting content straight to stdout" was a bit careless, because of
course you need to actively extract the links as above)

If this is all it takes to make it possible for ambitious client authors
to support word reflowing on mobile clients without making it impossible
to include ASCII art, source code or poetry, that seems like a fair
trade to me.

> Option 1. Use a no-op link
> 
> Pick a URL that by convention doesn't
> lead anywhere useful, and then hijack
> the (link friendly name) portion to
> display your gemini link.
> 
> =># =>/about.txt About
> 
> "#" is a valid relative URL, right?

This is kind of a cute hack, I'll admit, but I worry that (especially to
people that aren't intimiately familiar with RFC3986!) it's obscure and
less easy to remember than the back tick syntax familiar from several
other markup languages.  Also...

> This would be displayed on
> most gemini clients as:
> 
> =>/about.txt About

...it would display as something like:

[7] =>/about.txt About

in AV-98.  And with a button styling in Castor!  In general I think it's
nice if clients have a little leeway in choosing how they want to
present links.

> The line would be highlighted as a link
> (unless clients choose to handle this
> special case), 

Which I guess would take at *least* two extra lines of code, giving this
approach only a one line advantage over the ``` approach which also
facilitates reflowing text (this hack is just to make it possible to
display link syntax insie text/gemini documents, right?).

> Option 2. Use text/plain
> 
> ...
> 
> How you feel about this option likely
> depends on which side of the fence you
> fall on regarding text/gemini usage.
> Should text/gemini be used like HTML is
> on the web, with most content being
> written as gemini files? Or should it be
> more like gopher, where directories are
> type text/gemini but many people write
> their blog posts and other leaf
> documents as text/plain.
> 
> Lately I have been leaning more towards
> the second interpretation. Take another
> example: Instead of writing a python
> code snippet inline in a text/gemini
> document, what if you instead added a
> link to your code snippet and served it
> as "text/x-python"? This feels natural
> to me given that other media content
> like images also can't be displayed
> inline.

This is a conversation well worth having!  From my point of view, Gemini
very deliberately chose to cast aside Gopher's strict distinction
between menus and content, in part because so many people in Gopherspace
seemed to be disatisfied with it.  So, it seems kind of a shame to just
disregard this and go back to the gopher way.  Of course, people are
totally free to do so if they want to!  Neither the protocol nor I will
object.

It is true that there seem to be quite a lot of cases in the wild where
people are using text/gemini menus to link to collections of text/plain
files.  Sometimes this seems to be accidental, because the files
actually contain attempted Gemini links, e.g.:

gemini://tilde.black/users/fox/journal/20190831-starting-out.txt

Other times I suspect this is to facilitate straightforward bihosting on
Gopher and Gemini.

I presonally plan to write Gemini content primarily in text/gemini, and
in fact I'll probably decide how to distribute content between my Gopher
and Gemini sites based in part on how well different kinds of content
benefit from Gemini's ability to do in-line links.  Is anybody else
planning to do this?  Are the people currently not doing this doing so
out of conscious choice or just carrying over old Gopher habits?

As for using in-line links as a way to include un-flowed text, I did
think of this option (maybe somebody else already mentioned it in the
list?).  It would work, but I feel like the user experience would be
fairly clunky, especially for the combination of articles with frequent
short code snippets (like programming tutorials) and simple
terminal-based clients where going "back" to the text/gemini document
from the text/x-python document wouldn't return the reader to the same
point in the (potentially quite long) document from which they followed
the link.

I do take your point, though, that we're already forced into that clunky
paradigm for images and other non-textual media...

Ultimately I don't think I like either of these alternatives as much as
the ``` syntax, but I'm happy to hear others' thoughts...

Cheers,
Solderpunk

From aaron at ajanse.me  Wed Jan 15 20:47:27 2020
From: aaron at ajanse.me (Aaron Janse)
Date: Wed, 15 Jan 2020 12:47:27 -0800
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200115202137.GA21763@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
 <20200115202137.GA21763@SDF.ORG>
Message-ID: 

On Wed, Jan 15, 2020, at 12:21 PM, solderpunk wrote:
> On Wed, Jan 15, 2020 at 12:03:28AM -0500, Michael Lazar wrote:
> > 
> > Not to mention, pages like this [1]
> > display perfectly on my iphone using a
> > gemini-http proxy server. Regardless of
> > whether you choose to adopt the ```
> > mode, you're still going to need to
> > recommend a line length for authors to
> > hard wrap their text/gemini files at.
> > And I suggest that 40 is still worth
> > considering for this.
> 
> Yes, I definitely intend to include a recommendation that text/gemini
> content be hard wrapped

May I ask why?

Hard wrapping causes several issues:

1. If someone is using git, for example, to version control their gemini blog
   source code, they might have to re-wrap entire paragraphs in order to add
   one word to the beginning.

2. Many markdown documents have 500+ column lines (one paragraph per
   newline). If we require that Gemini source code is hard-wrapped, users
   couldn't copy-paste markdown into their gemini blog/site without
   hard-wrapping first.

In fact, I think *discouraging* hard-wrapping might actually make life easier
for client implementers. Hard-wrapping would require clients to un-wrap the
newlines then re-wrap to the desired width. I think that's more complex than:

1. Leaving wrapping up to the output medium (e.g. terminal, web browser).

2. Implementing wrapping within the client source code (which would be done
   anyway if Gemini source code *can be* hard-wrapped)

Oh, well. I'm vocal about this because Gemini looks really exciting to me.
As someone who plans to use it, I simply want it to be what I think is as
easy to use as possible :-)

Cheers!

From b__m__e at mailfence.com  Wed Jan 15 21:21:54 2020
From: b__m__e at mailfence.com (Brian Evans)
Date: Wed, 15 Jan 2020 22:21:54 +0100 (CET)
Subject: Text reflow woes (or: I want bullets back!)y
Message-ID: <1959862440.15925.1579123314431@ichabod.co-bxl>

> 2. Many markdown documents have 500+ column lines (one paragraph per
>    newline). If we require that Gemini source code is hard-wrapped, users
>    couldn't copy-paste markdown into their gemini blog/site without
>    hard-wrapping first.

Why would they need to hard wrap it? It would be the client or terminal that would be required to follow the spec and hard wrap it, not the content creators. I think every current client hard wraps at least to window size (if not some other value). This makes it so that a user can have a 500+ column row and the client/terminal/browser/etc will wrap it for them, leaving nothing for the user to do but paste it into their document for it to be wrapped automatically. I would image that most clients that allow the saving of pages would retain the original document structure free of any modifications made by the wrap (though I suppose mileage may vary in that regard).


This text reflow thread has been quite the minefield over time. Who would have thought that this would be the big thing to figure out when all this started?

From solderpunk at SDF.ORG  Wed Jan 15 22:11:17 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Wed, 15 Jan 2020 22:11:17 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: 
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
 <20200115202137.GA21763@SDF.ORG>
 
Message-ID: <20200115221117.GB21763@SDF.ORG>

On Wed, Jan 15, 2020 at 12:47:27PM -0800, Aaron Janse wrote:

> > Yes, I definitely intend to include a recommendation that text/gemini
> > content be hard wrapped
> 
> May I ask why?

Mostly because I want simple-as-possible clients to be viable, which
means simply printing non-link lines to stdout should result in
something usable.  Paragraphs of text formatted as a single line are
tremendously unpleasant to read when displayed this way!  If this were
the norm for text/gemini documents, I suspect nobody would use any
client that didn't include (tedious to write!) code to wrap these
lines at word breaks.

> In fact, I think *discouraging* hard-wrapping might actually make life easier
> for client implementers. Hard-wrapping would require clients to un-wrap the
> newlines then re-wrap to the desired width.

It wouldn't *require* that, the lines could simply be displayed at their
hard-wrapped width, which is why we're concerned with recommending a
narrow enough width that this would work on devices with narrow
displays.

Basically, consider what happens with the bare minimum amount of code in
either case:

A) If an entire paragraph is one line and a client doesn't have code to
break that big line up at word boundaries, leaving wrapping up to the
terminal results in multiple lines (potentially uncomfortably long for
reading) with randomly cut-up words at their beginnings and ends.

B) If the paragraph is hard-wrapped at ~40 characters and a client doesn't
have code to un-wrap and re-rewrap those lines, the result is lines of a
comfortabe length for reading, which fit on a smartphone and don't have any
randomly cut-up words at their beginnings and ends.

Given these two choices, surely B) is preferable?

Admittedly, this analysis is somewhat terminal-centric.  A lot of GUI
toolkits probably have text displaying widgets which will handle
breaking lines at word boundaries without the developer having to give
it a second though.

On the one hand, the vast majority of extant clients are
terminal-centric and I think the majority of the early adopters (being
gopher folk) lead terminal-centric lives, so a terminal-centric
perspective is only natural.  On the other hand, Gemini isn't supposed
to be only for a certain group of people, so I'm reluctant to lean on
this too much...

> Oh, well. I'm vocal about this because Gemini looks really exciting to me.
> As someone who plans to use it, I simply want it to be what I think is as
> easy to use as possible :-)

I appreciate you speaking up and I'm glad you're excited!

Cheers,
Solderpunk

From solderpunk at SDF.ORG  Wed Jan 15 22:12:57 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Wed, 15 Jan 2020 22:12:57 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <1959862440.15925.1579123314431@ichabod.co-bxl>
References: <1959862440.15925.1579123314431@ichabod.co-bxl>
Message-ID: <20200115221257.GC21763@SDF.ORG>

On Wed, Jan 15, 2020 at 10:21:54PM +0100, Brian Evans wrote:

> Who would have thought that this would be the big thing to figure out when all this started?

Let me assure you, I certainly didn't! :)

Cheers,
Solderpunk

From aaron at ajanse.me  Thu Jan 16 02:41:00 2020
From: aaron at ajanse.me (Aaron Janse)
Date: Wed, 15 Jan 2020 18:41:00 -0800
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200115221117.GB21763@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
 <20200115202137.GA21763@SDF.ORG>
 
 <20200115221117.GB21763@SDF.ORG>
Message-ID: <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com>

On Wed, Jan 15, 2020, at 2:11 PM, solderpunk wrote:
> B) If the paragraph is hard-wrapped at \~40 characters and a client doesn't
> have code to un-wrap and re-rewrap those lines, the result is lines of a
> comfortabe length for reading, which fit on a smartphone and don't have any
> randomly cut-up words at their beginnings and ends.
> 
> Given these two choices, surely B) is preferable?
> 

I think that one-line-per-paragraph would be much better for mobile phones.
I personally use a web browser proxy to read gemini pages when on mobile.
Text wrapped at 80 columns looks horrible. Text wrapped at 40 columns looks
okay, depending on my font size. But if text was one-line-per-paragraph,
firefox would wrap it wonderfully with *zero* effort from the proxy author.

> Admittedly, this analysis is somewhat terminal-centric.  A lot of GUI
> toolkits probably have text displaying widgets which will handle
> breaking lines at word boundaries without the developer having to give
> it a second though.

Yeah. And the variety of clients adds a lot of nuance to how text wrapping
degrades.

I can only imagine unlimited-column text catastrophically failing in two
places:

1. A very wide terminal
2. A very wide web browser

However, each has a simple solution:

1. Pipe text into the bash `fmt` command, which intelligently wraps text
2. Use *simple* css to make the viewport narrower, then let the browser
   do the wrapping

Thanks, solderpunk, for being a thoughtful BDFL!

Cheers!


From lazar.michael22 at gmail.com  Thu Jan 16 04:58:48 2020
From: lazar.michael22 at gmail.com (Michael Lazar)
Date: Wed, 15 Jan 2020 23:58:48 -0500
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <20200115202137.GA21763@SDF.ORG>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
 <20200115202137.GA21763@SDF.ORG>
Message-ID: 

On Wed, Jan 15, 2020 at 3:21 PM solderpunk  wrote:
>
> On Wed, Jan 15, 2020 at 12:03:28AM -0500, Michael Lazar wrote:
> > Here are some other alternatives that
> > might be worth considering. I do think
> > that displaying gemini links is a valid
> > use-case, but adding a whole new
> > preformatted text mode only for this
> > narrow case feels a bit heavy-handed to
> > me. Granted, I realize there are other
> > benefits to the preformatted mode that
> > have already been outlined.
>
> ...
>
> Ultimately I don't think I like either of these alternatives as much as
> the ``` syntax, but I'm happy to hear others' thoughts...
>
> Cheers,
> Solderpunk

Thanks for your response. After seeing
everything laid out in front of me, I
think I agree with you and I like your
proposal the best.

I wanted to visualize what this would
look like on mobile, so  I hacked up my
Gemini-HTTPS proxy to render the ```
markdown syntax. Here are some example
pages:

https://portal.mozz.us/gemini/mozz.us/files/aaatutorial.gmi?reflow=2
https://portal.mozz.us/gemini/mozz.us/files/flask_mega_tutorial_part_2.gmi?reflow=2

I think they look quite nice!

- mozz

From solderpunk at SDF.ORG  Thu Jan 16 15:10:01 2020
From: solderpunk at SDF.ORG (solderpunk)
Date: Thu, 16 Jan 2020 15:10:01 +0000
Subject: Text reflow woes (or: I want bullets back!)y
In-Reply-To: <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com>
References: <1367854366.219683.1576461212912@ichabod.co-bxl>
 
 <20191216054032.GD24166@brevard.conman.org>
 <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com>
 <20200112184341.GA1011@SDF.ORG>
 
 <20200115202137.GA21763@SDF.ORG>
 
 <20200115221117.GB21763@SDF.ORG>
 <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com>
Message-ID: <20200116151001.GA24028@SDF.ORG>

On Wed, Jan 15, 2020 at 06:41:00PM -0800, Aaron Janse wrote:
 
> I think that one-line-per-paragraph would be much better for mobile phones.
> I personally use a web browser proxy to read gemini pages when on mobile.
> Text wrapped at 80 columns looks horrible. Text wrapped at 40 columns looks
> okay, depending on my font size. But if text was one-line-per-paragraph,
> firefox would wrap it wonderfully with *zero* effort from the proxy author.

I am kind of reluctant to make any Gemini design decisions based on the
assumption of a web browser as the user agent.  I understand that in
these very early days this is by far the quickest and easiest way to get
into Geminispace, and for smartphones it's probably the *only* viable
way.  But I hope that as time moves forward the proxies will become a
niche thing, serving as a "gateway drug" for real clients.  This is
surely a valuable role, but for regular use I think they should be
"considered harmful".  They represent single (or few) points of failure,
they involve trusting the proxy operator not to manipulate content
(Gemini isn't sophisticated enough to permit proper TLS proxying, so the
the web proxy is basically a MITM between client and server), proxy
operators have more opportunities to log and track their users than
individual Gemini servers do, and proxy users need to run vastly more
complex than necessary software (i.e. web browsers, although at least
Gemini proxies should typically be usable with nice alternative browsers
like dillo).

(aside: some of these issues go away with proxies designed to be run
locally by the user - a nice project for anybody itching for one!)

None of this is to say the web proxies or bad and that people shouldn't
run them or use them - I'm very gratefully to the people who have set
them up!  But I don't think of them as "first class" clients, and given
a choice between pushing implementation effort onto native client
authors or onto web proxy authors, I will make life easier for native
client authors every time.

Besides, getting a web proxy to provide beautiful wrapping if a
text/gemini file is hard-wrapped at 40 chars involves nothing more than
wrapping paragraphs in 

and

tags. That's a significantly easier task than getting a terminal client to provide beautiful wrapping if a text/gemini file has lines thousands of characters long, which requires splitting the line into words, calculating and summing the lengths of words, etc, etc. Given the choice between making web proxy authors do a little bit more work and makig native client authors to a moderate amount of more work, I'm definitely going to choose the former. > I can only imagine unlimited-column text catastrophically failing in two > places: > > 1. A very wide terminal > 2. A very wide web browser Neither of which are terribly uncommon, right, with full-screen windows on desktops or even laptops? My "daily driver" laptop terminal is 113 chars wide. Although, even on a terminal < 80 chars wide, I kind of consider words being split across lines as pretty severe failure. I don't want to read that - it's even less pleasant than hard-wrapped 80 char lines on a mobile. > Thanks, solderpunk, for being a thoughtful BDFL! I'm glad you think I'm thoughtful! Sorry if I seem to be dismissing the "long lines" approach out of hand, I promise you I'm giving it a lot of thought. I'm already stressing out that I'm being unduly influenced by the fact that I use simple / old-fashioned editors and mostly write stuff that should be hard-wrapped (plain text email, gopher content, source code). From my perspective, writing "long line" content is a less pleasant experience for authors, because my editors don't work that way out of the box. But I realise that, actually, for the majority of people that's far *more* accessible. Someone using something resembling Notepad is going to have a miserable time writing content hard-wrapped at 40 chars, while the "long line" format just happens, probably without them even realising it. Then again, making text/gemini easy to write with "normal" editors arguably isn't worth much if the next step is anyway "now use sftp or a git push to get your content on the server". Gemini is never going to be able to support easy WYSIWIG authoring experiences akin to WordPress, so perhaps it's pointless to consider the user experience for non-technical types. Argh! Simplicity ain't simple. Cheers, Solderpunk From aaron at ajanse.me Thu Jan 16 17:40:50 2020 From: aaron at ajanse.me (Aaron Janse) Date: Thu, 16 Jan 2020 09:40:50 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116151001.GA24028@SDF.ORG> References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> Message-ID: On Thu, Jan 16, 2020, at 7:10 AM, solderpunk wrote: > But I realise that, actually, for the majority of people that's far > *more* accessible. Someone using something resembling Notepad is going > to have a miserable time writing content hard-wrapped at 40 chars, while > the "long line" format just happens, probably without them even realising > it. I do admit that you've nearly convinced me that 40 chars is easiest for readers, even though I'm not sure if it's best if we consider authors, too. > Then again, making text/gemini easy to write with "normal" editors > arguably isn't worth much if the next step is anyway "now use sftp or a > git push to get your content on the server". In my experience, though, hard-wrapping text doesn't work well with git anyway. And I don't know of any text editors that re-hard-wrap automatically when the beginning of a paragraph is edited. Maybe gemini files could be authored in either wrapped or unwrapped format, then the server could hard-wrap intelligently before sending it to the reader? Would that be bad practice? Cheers! From sean at conman.org Thu Jan 16 18:27:46 2020 From: sean at conman.org (Sean Conner) Date: Thu, 16 Jan 2020 13:27:46 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> Message-ID: <20200116182746.GA11266@brevard.conman.org> It was thus said that the Great Aaron Janse once stated: > On Thu, Jan 16, 2020, at 7:10 AM, solderpunk wrote: > > But I realise that, actually, for the majority of people that's far > > *more* accessible. Someone using something resembling Notepad is going > > to have a miserable time writing content hard-wrapped at 40 chars, while > > the "long line" format just happens, probably without them even realising > > it. > > I do admit that you've nearly convinced me that 40 chars is easiest for > readers, even though I'm not sure if it's best if we consider authors, too. > > > Then again, making text/gemini easy to write with "normal" editors > > arguably isn't worth much if the next step is anyway "now use sftp or a > > git push to get your content on the server". > > In my experience, though, hard-wrapping text doesn't work well with git > anyway. And I don't know of any text editors that re-hard-wrap automatically > when the beginning of a paragraph is edited. > > Maybe gemini files could be authored in either wrapped or unwrapped format, > then the server could hard-wrap intelligently before sending it to the > reader? Would that be bad practice? First question---how to tell the server the width? Well, one solution: gemini://gemini.conman.org/test/wrap;80 gemini://gemini.conman.org/test/wrap;40 gemini://gemini.conman.org/test/wrap;32 gemini://gemini.conman.org/test/wrap;132 Just replace the number at the end with your preferred width. If not given: gemini://gemini.conman.org/test/wrap it will currently default to 77. I just happen to have the code to handle reflowing text at arbitrary widths, but the code to do so has to take into account a few edge cases that might not be readily apparent (what if there is no breakpoint within the mandated width?). -spc (Also note---I added that to my Gemini site last September) From aaron at ajanse.me Thu Jan 16 19:48:09 2020 From: aaron at ajanse.me (Aaron Janse) Date: Thu, 16 Jan 2020 11:48:09 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116182746.GA11266@brevard.conman.org> References: <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <20200116182746.GA11266@brevard.conman.org> Message-ID: <4e1d76cb-eff5-4ecb-9470-dbc22860d89f@www.fastmail.com> On Thu, Jan 16, 2020, at 10:27 AM, Sean Conner wrote: > First question---how to tell the server the width? Well, one solution: On one hand, I like the idea of the server doing the wrapping. It would allow viewport-specific fixed text. For example, two things that server-side wrapping would allow that are currently used in the Gemini speculative spec: - viewport-width-specific dividers (a bunch of dashes) - indented paragraphs or trailing-indent bullet points On the other hand, server-side wrapping has the following drawbacks: - broadcasting viewport width goes against Gemini privacy values - clients can no longer download a gemini blog then view it at different widths - this could open up room to server-side trickery, such as indenting fixed-width content, which I don't think is a good idea Also, speaking of privacy, does Gemini work over Tor? From lel at envs.net Thu Jan 16 20:26:40 2020 From: lel at envs.net (lel) Date: Thu, 16 Jan 2020 15:26:40 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <4e1d76cb-eff5-4ecb-9470-dbc22860d89f@www.fastmail.com> References: <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <20200116182746.GA11266@brevard.conman.org> <4e1d76cb-eff5-4ecb-9470-dbc22860d89f@www.fastmail.com> Message-ID: <20200116202640.GA25554@briz> On Thu, Jan 16, 2020 at 11:48:09AM -0800, Aaron Janse wrote: > On Thu, Jan 16, 2020, at 10:27 AM, Sean Conner wrote: > > First question---how to tell the server the width? Well, one solution: > > On one hand, I like the idea of the server doing the wrapping. > > It would allow viewport-specific fixed text. For example, two things > that server-side wrapping would allow that are currently used in the > Gemini speculative spec: > > - viewport-width-specific dividers (a bunch of dashes) > - indented paragraphs or trailing-indent bullet points > > On the other hand, server-side wrapping has the following drawbacks: > > - broadcasting viewport width goes against Gemini privacy values > - clients can no longer download a gemini blog then view it at > different widths > - this could open up room to server-side trickery, such as indenting > fixed-width content, which I don't think is a good idea > > Also, speaking of privacy, does Gemini work over Tor? Yeah, I don't think this does it because not only does it complicate server implementation, but it does so in a way that seeks to allow servers to learn about clients. That being said, as far as I'm concerned, literally anything at all is preferable to hard-wrapping at 40 characters -- less than a quarter of my *laptop* screen with a mid-size font; I can only imagine how it looks on an actual monitor. Limiting lines to 5-6 words would seem to limit, on a fundamental level, the types of content that can even be served over gemini. Adding an arbitrary cap to line lengths purely so that a hypothetical mobile client doesn't require 10-20 lines of wrapping code (code that Google suggests already exists in the Android SDK and merely has to be invoked) seems absurd to me, particularly when the wrapping itself is trivial, and this is entirely because word-wrapping is considered preferable to naive, occasionally-mid-word wrapping. The argument I'm reading is that nobody would use a client that occasionally wraps in the middle of words, and the gemini spec is explicitly geared toward making client implementations as simple as possible. I understand this, yet I think there exists a world of difference between "it should be trivial to write a client" and "it should be trivial to write a client that formats every line beautifully and that we would all love using," particularly when the point of difference between the two is whether or not some words are line-broken some of the time. This is just my two cents, but I feel like that's something that it makes more sense to leave up to the client. The actual client-side process of interacting with the server is still simple, but printing it can be done in a simple way (wrap, occasionally breaking words) or with a few lines and a couple conditions to guard edge-cases (to word-wrap). I don't know, I understand that some arguments in favor of a 40 character hard-wrap have been made from a reading level, and I understand them, but I don't think that justifies hard-capping all lines at only a few words. If you want to be able to speed-read lines, you could write a client that word-wraps at 40 characters very, very easily. You could write a client that flashes words at you one at a time quickly like those speed-reading programs if you want to take that idea to an extreme. But I feel like this is a very drastic response to a non-issue. Maybe I wouldn't feel so bad about it if the limit being suggested weren't as puny as 40 characters. But no matter what the limit is, there are still screens, or fonts, or what have you, where any hard-wrapping solution does not solve the problem. And if aesthetics are the main concern, and we can't stomach a few split words, then I really don't get why the best solution is to squash every single gemini site into the left 25% of our screens. I'm sure whatever flagship gemini client eventually emerges on top is going to be beautiful and not wrap in the middle of words, but I also don't think that the spec should be specced based upon the hypothetical flagship client -- client implementation can be simple, and it will no doubt remain so for gemini; but flagship client implementation is not, cannot, and will never be, for any protocol. I'm not trying to be insulting to anyone, and I get that y'all will likely disagree. I fully understand the reasons behind what everyone in favor of the 40 char limit is suggesting, but I think that practically speaking, it's not liveable for anyone trying to use gemini, and it causes more problems than it solves. idk ~ lel From solderpunk at SDF.ORG Thu Jan 16 20:26:52 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Thu, 16 Jan 2020 20:26:52 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> Message-ID: <20200116202652.GA23953@SDF.ORG> On Thu, Jan 16, 2020 at 09:40:50AM -0800, Aaron Janse wrote: > In my experience, though, hard-wrapping text doesn't work well with git > anyway. And I don't know of any text editors that re-hard-wrap automatically > when the beginning of a paragraph is edited. Hmm. Neither do I and, in fact, well...I just do it manually. Which I feel very sheepish admitting because that's kind of ridiculous. But everybody writing for Gopherspace (which is many people here) must face precisely this problem, because hard-wrapping is basically compulsory there. What are other people doing, writing in "long line" form and then feeding the result to `fmt` or `par` before uploading? Cheers, Solderpunk From lazar.michael22 at gmail.com Thu Jan 16 20:38:23 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Thu, 16 Jan 2020 15:38:23 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> Message-ID: On Thu, Jan 16, 2020 at 12:41 PM Aaron Janse wrote: > > On Thu, Jan 16, 2020, at 7:10 AM, solderpunk wrote: > > Then again, making text/gemini easy to write with "normal" editors > > arguably isn't worth much if the next step is anyway "now use sftp or a > > git push to get your content on the server". > > In my experience, though, hard-wrapping text doesn't work well with git > anyway. And I don't know of any text editors that re-hard-wrap automatically > when the beginning of a paragraph is edited. > > Maybe gemini files could be authored in either wrapped or unwrapped format, > then the server could hard-wrap intelligently before sending it to the > reader? Would that be bad practice? > > Cheers! I see no reason we can't let each server decide this on a case-by-case basis, based on the needs of their users. If you're hosting shared gemini content for a group of non-technical users, it might make sense for your server to automatically reformat gemini files before sending them. On the other hand, if you're running a home-grown gemini server and writing all of your own content, this step might be unnecessary or invasive to your preferred workflow. If we're adopting the proposed format with ``` for preformatted blocks, the rules can be totally ambiguous for server-side line wrapping. 1. preformatted blocks are not touched by the server 2. links are not touched by the server 3. everything else can be wrapped by the server to the preferred width Content authors should not expect their non preformatted text to have line lengths / indents / etc. preserved. - mozz From aaron at ajanse.me Thu Jan 16 20:50:45 2020 From: aaron at ajanse.me (Aaron Janse) Date: Thu, 16 Jan 2020 12:50:45 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> Message-ID: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> On Thu, Jan 16, 2020, at 12:26 PM, solderpunk wrote: > But everybody writing for Gopherspace (which is many people here) must > face precisely this problem, because hard-wrapping is basically compulsory > there. And my understanding is that while Gemini isn't supposed to be a Gopher 2.0, it's supposed to make it better than Gopher. I think that non-hard-wrapping could be a big step towards this goal. lel said: > Adding an arbitrary cap to line lengths purely so that a hypothetical > mobile client doesn't require 10-20 lines of wrapping code (code that > Google suggests already exists in the Android SDK and merely has to be > invoked) seems absurd to me, particularly when the wrapping itself is > trivial, and this is entirely because word-wrapping is considered > preferable to naive, occasionally-mid-word wrapping. Honestly, even if it takes 40 lines of code, I'd rather write that then manually wrap every single gemini document I ever write. > What are other people doing, writing in "long line" form and > then feeding the result to `fmt` or `par` before uploading? For things such as email, after manually wrapping quotes, I think using `fmt` or `par` is the most efficient way to wrap text. However, this only works for text that's written once, such as emails. This doesn't work for continually edited content such as websites. > If you're hosting shared gemini content for a group of non-technical users, > it might make sense for your server to automatically reformat gemini files > before sending them I don't have a problem with people "rendering" into Gemini. I just think that the spec shouldn't make it necessary. Rendering/compiling into Gemini adds a level of indirection that hides the transparency that makes me love the protocol. Cheers! From solderpunk at SDF.ORG Thu Jan 16 20:54:38 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Thu, 16 Jan 2020 20:54:38 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> Message-ID: <20200116205438.GB23953@SDF.ORG> On Wed, Jan 15, 2020 at 11:58:48PM -0500, Michael Lazar wrote: > I wanted to visualize what this would > look like on mobile, so I hacked up my > Gemini-HTTPS proxy to render the ``` > markdown syntax. Here are some example > pages: > > https://portal.mozz.us/gemini/mozz.us/files/aaatutorial.gmi?reflow=2 > https://portal.mozz.us/gemini/mozz.us/files/flask_mega_tutorial_part_2.gmi?reflow=2 > > I think they look quite nice! These look great! And I don't just mean aesthetically, they are an excellent example of the practical usefulness of being able to embed prefromatted text in otherwise "normal" prose. Cheers, Solderpunk From aaron at ajanse.me Thu Jan 16 21:09:18 2020 From: aaron at ajanse.me (Aaron Janse) Date: Thu, 16 Jan 2020 13:09:18 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> Message-ID: <546c2284-4b16-4ec1-8a9f-a072e0773799@www.fastmail.com> Oops, I meant to mention two things: 1. Clients are already using TLS. I think wrapping is significantly easier than encryption. 2. Lists are tough to handle when re-flowing text. Thanks again for the thoughtfulness, solderpunk! From lazar.michael22 at gmail.com Thu Jan 16 22:00:59 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Thu, 16 Jan 2020 17:00:59 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> References: <1367854366.219683.1576461212912@ichabod.co-bxl> <20191216054032.GD24166@brevard.conman.org> <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> Message-ID: On Thu, Jan 16, 2020 at 3:51 PM Aaron Janse wrote: > On Thu, Jan 16, 2020, at 12:26 PM, solderpunk wrote: > > If you're hosting shared gemini content for a group of non-technical users, > > it might make sense for your server to automatically reformat gemini files > > before sending them > > I don't have a problem with people "rendering" into Gemini. I just think that > the spec shouldn't make it necessary. Rendering/compiling into Gemini adds > a level of indirection that hides the transparency that makes me love the > protocol. > > Cheers! To be clear, I agree with you that a max line length shouldn't be enforced by the spec. I support the current wording of SHOULD as opposed to MUST. I'm more talking about establishing a recommended best practice. Gemini files with lines over N characters long should still be considered "valid" and must be accepted by any gemini client. They might just look a little funky depending on how sophisticated your client is. My belief is that if we don't establish a guidance on the line length, the community will informally gravitate towards one anyway. And that will almost certainly be 70-80 characters because that's what most people in this community are used to. Servers that return 500+ characters on a single line will look bad on many of the gemini clients that people write, and content authors will be pressured into conforming to maintain readability for those clients. The spec-spec as written right now says that all gemini lines should be formatted so that they can reflowed by clients if the client chooses to do so. That spec has been almost unanimously rejected by current gemini servers, in favor of hard wrapping at 70-80 characters. - mozz From solderpunk at SDF.ORG Thu Jan 16 22:12:02 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Thu, 16 Jan 2020 22:12:02 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <546c2284-4b16-4ec1-8a9f-a072e0773799@www.fastmail.com> References: <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <546c2284-4b16-4ec1-8a9f-a072e0773799@www.fastmail.com> Message-ID: <20200116221202.GC23953@SDF.ORG> On Thu, Jan 16, 2020 at 01:09:18PM -0800, Aaron Janse wrote: > Oops, I meant to mention two things: > 1. Clients are already using TLS. I think wrapping is significantly > easier than encryption. I was about to quip that the TLS is made really easy by high-level library support while the wrapping has to be done manually, but then I thought to check and, holy heck, Python has a `textwrap` module I wasn't aware of. > 2. Lists are tough to handle when re-flowing text. One of the most appealing things to me about the "long lines" approach is that it actually makes lists with multi-line items feasible without having to define any special syntax for them. I might elaborate on this later... Cheers, Solderpunk From sean at conman.org Thu Jan 16 22:15:29 2020 From: sean at conman.org (Sean Conner) Date: Thu, 16 Jan 2020 17:15:29 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116202652.GA23953@SDF.ORG> References: <86d7af23-fc3d-49e6-9bd6-f68c84fe8899@www.fastmail.com> <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <20200116202652.GA23953@SDF.ORG> Message-ID: <20200116221529.GA3354@brevard.conman.org> It was thus said that the Great solderpunk once stated: > On Thu, Jan 16, 2020 at 09:40:50AM -0800, Aaron Janse wrote: > > > In my experience, though, hard-wrapping text doesn't work well with git > > anyway. And I don't know of any text editors that re-hard-wrap automatically > > when the beginning of a paragraph is edited. > > Hmm. Neither do I and, in fact, well...I just do it manually. Which I > feel very sheepish admitting because that's kind of ridiculous. But > everybody writing for Gopherspace (which is many people here) must face > precisely this problem, because hard-wrapping is basically compulsory > there. What are other people doing, writing in "long line" form and > then feeding the result to `fmt` or `par` before uploading? Setting aside my phlog [1], the rest of the content on my gopher and Gemini servers is written with 80 columns in mind. The editor I use will wrap at a default setting of 77 (but can be changed on the fly). The last computer I used with a width of less than 80 characters was my TRS-80 Color Computer which had a width of 32 characters (and I last used regularly in 1987). Since then, all the computers I've had supported at least 80 characters [2]. For my phlog, it's a rendering of my web-based blog [3] where I use Lynx to do the conversion from HTML to plain text (with a bit of post-processing to fix intra-blog links). How do I write my blog entries? I use my regular editor and I've adopted a style a few years ago where I write ... not exactly a sentance per line, but a thought per line. Okay, an example from a previous post [4]: [=== example ===]

Okay, to be fair, I did find references and draft material covering the problem of pirates, but I found his stance on a 12 gauge shotgun to be ?more accurate? than a hand gun to be questionable at best. ?Accuracy? on a rolling, pitching boat in the open water is going to be questionable, regardless of choice of firearm.

There is correspondence with yatch manufactuers, blue prints, price breakdowns (nearly $300,000 in 1982 dollars, making it nearly $800,000 in today's dollars?ouch!) and scores of articles on everything related to sailing. It also appears that Dad was trying to invent a new type of sail, as there were drawings he did and correspondence with an engineering firm. I'm not sure what I'll do with it all, but the blueprints are cool.

[=== end ===] Web browsers don't care about the raw formatting---they reflow it, with styling coming from HTML and CSS. And the reason I keep it very ragged like that is to make it easier to move sentances and fragments about when editing the entry. I also run the entry through a processing script to convert some shortcuts, like turning `` into ? or "1^st" into "1ST" [4] or even pulling out image sizes for the tag. This is probably more than you wanted to know 8-) -spc [1] gopher://gopher.conman.org/1phlog.gopher [2] Okay, yes! I have an iPhone. I don't use it to browse gopherspace though. [3] http://boston.conman.org/ [4] Yes, I have a type of markup I use, but it's totally custom to me and how I write entries, but it does quite a bit. Here's a sample that includes all the features (I think I keep this up to date): https://github.com/spc476/mod_blog/blob/master/NOTES/testmsg And the script I use: https://github.com/spc476/mod_blog/blob/master/Lua/format.lua From sean at conman.org Thu Jan 16 22:19:23 2020 From: sean at conman.org (Sean Conner) Date: Thu, 16 Jan 2020 17:19:23 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116221529.GA3354@brevard.conman.org> References: <20200112184341.GA1011@SDF.ORG> <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <20200116202652.GA23953@SDF.ORG> <20200116221529.GA3354@brevard.conman.org> Message-ID: <20200116221922.GB3354@brevard.conman.org> It was thus said that the Great Sean Conner once stated: > > How do I write my blog entries? I use my regular editor and I've adopted > a style a few years ago where I write ... not exactly a sentance per line, > but a thought per line. Okay, an example from a previous post [4]: This footnote [4] should have pointed to here: http://boston.conma.norg/2020/01/13.1 > Web browsers don't care about the raw formatting---they reflow it, with > styling coming from HTML and CSS. And the reason I keep it very ragged like > that is to make it easier to move sentances and fragments about when editing > the entry. I also run the entry through a processing script to convert some > shortcuts, like turning `` into ? or "1^st" into "1ST" [4] or > even pulling out image sizes for the tag. And this footnote [4] should have been footnote [5]. I got distracted when writing the email. -spc > [5] Yes, I have a type of markup I use, but it's totally custom to me > and how I write entries, but it does quite a bit. Here's a sample > that includes all the features (I think I keep this up to date): > > https://github.com/spc476/mod_blog/blob/master/NOTES/testmsg > > And the script I use: > > https://github.com/spc476/mod_blog/blob/master/Lua/format.lua From solderpunk at SDF.ORG Thu Jan 16 22:24:58 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Thu, 16 Jan 2020 22:24:58 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> Message-ID: <20200116222458.GD23953@SDF.ORG> On Thu, Jan 16, 2020 at 05:00:59PM -0500, Michael Lazar wrote: > To be clear, I agree with you that a max line length shouldn't be enforced by > the spec. I support the current wording of SHOULD as opposed to MUST. I'm more > talking about establishing a recommended best practice. Gemini files with lines > over N characters long should still be considered "valid" and must be accepted > by any gemini client. They might just look a little funky depending on how > sophisticated your client is. Oh, absolutely. All this discussion about hard wrapping at 40 chars is firmly in the real of best practice recommendation. > The spec-spec as written right now says that all gemini lines should be > formatted so that they can reflowed by clients if the client chooses to do so. > That spec has been almost unanimously rejected by current gemini servers, in > favor of hard wrapping at 70-80 characters. The current spec-spec says that clients can reflow long lines if they want to, but it doesn't place any obligation on authors or servers to actually provide long lines. Cheers, Solderpunk From aaron at ajanse.me Thu Jan 16 23:26:13 2020 From: aaron at ajanse.me (Aaron Janse) Date: Thu, 16 Jan 2020 15:26:13 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116222458.GD23953@SDF.ORG> References: <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> Message-ID: <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> On Thu, Jan 16, 2020, at 2:24 PM, solderpunk wrote: > The current spec-spec says that clients can reflow long lines if they > want to, but it doesn't place any obligation on authors or servers to > actually provide long lines. Hmm, even allowing reflowing sounds like it could cause incompatibility issues. Should there be standardized markup for bullet points? What about poems, something I think are much easier to find on Gopher than on the HTML web? How about markdown title lines immediately followed by text? I think that wrapping then reflowing fundamentally leads to loss of original information. I'll provide below a few things that I'm not sure how to reflow. Note that, except for the haiku, many of the lines could go past the viewport width, making fixed text mode not an option ( especially for lists). This is a haiku Should it be reflowed or not? I don't think it should. - this is a bullet point - so is this * and this * and this * and this -> and this 1. and this I. and this II. and this ~ and this + and using plus signs is even valid markdown *R*emember *A*crostic *P*oems One word per line for emphasis # Title Start of paragraph immediately after ## Subtitle without spacing above > Quote for this sub-section While providing the option of hard-wrapped text would appeal to retro users, it could come at the cost of a much more complex spec for how to reflow along with incompatibilities as clients act differently when confronted with un-specified situations. Cheers From solderpunk at SDF.ORG Fri Jan 17 11:11:58 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Fri, 17 Jan 2020 11:11:58 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> References: <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> Message-ID: <20200117111158.GA10236@SDF.ORG> On Thu, Jan 16, 2020 at 03:26:13PM -0800, Aaron Janse wrote: > > Hmm, even allowing reflowing sounds like it could cause incompatibility > issues. Should there be standardized markup for bullet points? What about > poems, something I think are much easier to find on Gopher than on the > HTML web? How about markdown title lines immediately followed by text? Well, a lot of this is exactly what the recently proposed ``` was supposed to help fix. Things precisely like the haiku example you provided would be enclosed within ``` lines so that they didn't get mangled by a reflowing client. A speedy recap of the history of text/gemini, excluding the link syntax debate which is thankfully past us: * Originally, the spec-spec said nothing about wrapping or flowing or anything of the sort. Everybody was producing hard-wrapped content and/or writing clients which expected hard-wrapped content (probably running on auto-pilot from Gopher experiece), so that was in some sense the implicit standard. * Several people wrote to me and said Gopher's hard line wrapping was one of their least favourite parts of Gopher because the days where everybody was at an 80 char terminal are over and Gopher content often looks mangled on their smartphones. * This seemed reasonable and easy to fix, so in what was now clearly a mistake I updated the spec-spec to its current version, permitting clients to reflow if they want to, because, hey, what's the worst that could happen? * Very shortly after the first client implementation efforts happened it became clear that this was a disaster, precisely because it breaks bulleted lists (hence the subject of this thread), but also other content like ASCII art, poems, source code, etc. * Everybody agreed there had to be some way to preserve these things, so the hunt was on for a slightly extended syntax to allow clearly signalling to reflowing clients "thou shalt not reflow/wrap/mangle these particular lines" (even at this point, there were voices which said this was all getting to hard and we should forget about reflowing). * After a false start involving the idea of leading and/or trailing whitespace signalling that text should not be reflowed (which I now think is a bad idea for several reasons and don't want us to consider again), the ``` line idea seems to have provoked no serious objections. I thought there was light at the end of the tunnel.. But as your email makes clear, the ``` syntax is not a panacea. It will allow haikus, one-word-per-line text, source and other things, so it's an improvement over the current spec-spec. However, it still doesn't really work, or works in a very inconvenient and ugly way, for many other things like lists, Markdown headings, etc. I don't hate bulleted lists, or Markdown headings. On the contrary, I like them and use them a lot and it would be lovely if Gemini documents could include them. But I really, really don't want: 1. To distract everybody's time and energy for the Gemini project for even more months than we already have hammering this stuff out when we could be discussing way more interesting and important stuff, or writing software, or best of all writing content so this thing takes off. 2. To end up with a final spec where the section that explains precisely how to render a text/gemini document involves more words than the sections which explain the actual protocol itself. text/gemini is supposed to be a lightweight format! 3. To end up with a text/gemini specification which is basically exactly the same as Markdown except with a different link syntax. That's just using a lot of time to basically reinvent a wheel, and it's an especially poor use of time when Gemini is already capable of just serving actual Markdown with a MIME type to indicate that it's doing that! And it sucks for authors of non-graphical clients, because how to properly render Markdown isn't actually defined beyond how to properly transform it to HTML, and how to render HTML in a terminal is not defined at all and there are few libraries for it. It would be an epic fail if simple terminal Gemini clients had to hand content over to lynx as a external formatting tool. 4. To compromise on the idea that writing a basic but usable client should be really quick and easy. I don't mind if text/gemini has some voluntary extra complexity. I really like the two digit status code solution we chose for the response header - extra information/power is there for when it is needed, but simple clients can ignore it and still function correctly. This has to be true for handling text/gemini too. Whatever extra complexity we add has to degrade gracefully when ignored. I get that really comfy, full-featured "flagship" clients are inevitably going to be more complicated, and that's fine. But I don't want to repeat the mistake of the web where flagship clients are basically compulsory, and if you try to take something nice and simple like Dillo outside of a very, very small subset of the web (maintained by people like us) you end up with a badly broken user experience. Something that a single person can bang out on the same weekend they read the spec for the first time should be able to render the majority of Geminispace into something that's comfortable to read. Not necessarily perfect, but something that you might voluntarily use to read content for half an hour. I just don't see how to simultaneously avoid all these things *and* end up with a text/gemini that nicely supports headings and lists and quotations which are robust against reflowing/wrapping. I honestly think it's impossible to simultaneously satisfy all three of: 1. Easy to describe and to parse 2. Capable of "richness" like headings, lists, quotes, etc. 3. Looks good on arbitrary screen widths Basically, you have to pick two. If you really want 2. and 3. you have no choice but define a whole set of special syntax like in HTML or Markdown (which is really just a more syntactically lightweight form of a subset of HTML), and that kills 1. Specifying a fixed character width for hard-wrapped lines gives you 1. and 2. (this is exactly how Gopher works, where people use all that richness all the time without even thinking about it), but breaks 3 because the fixed character width is always going to be too long or too short for some device. And finally, the current spec-spec delivers 1. and 3. It's not hard to reflow just plain paragraphs of text to arbitrary widths, the breakage only occurs when the text contains richness like headers, lists, quotes, etc. By adding ``` to the spec-spec we can allow a tiny bit of richness in (poems, source code, ASCII art), but it's not enough to give us anything like a web-like or even gopher-like textual experience. Compromising on 1. means compromising on the entire point of Gemini, so that's not happening. We need to either compromise on 2. by giving up on the idea that text/gemini documents can reliably support headings, lists, etc. (and this will be a bit of a blow for the idea of boostrapping Geminispace by mirroring existing Gopher content, because a lot of that Gopher content is going to include exactly those things in Markdownesque format without any Gemini-specific ``` lines it, and so will be mangled by reflowing Gemini clients) or we need to compromise on 3. by defining a fixed character width (with doesn't necessarily have to be 40) and accepting that Gemini just won't look great on tiny screens. Right now I'm inclined to sacrifice 3. more than I am 2. Not working nicely on smartphones will be a shame and probably upset a lot of people, but *something* has to give and it seems sensible for that to be devices which, at the end of the day, are obviously not well suited to displaying non-trivial quantities of text anyway. There's a reason that newspapers, magazines, books, e-readers and even folded brochures are all wider than phones! And it would just be embarrassing for Gemini documents to need to be structurally simpler than Gopher documents (by never using lists, etc.). > While providing the option of hard-wrapped text would appeal to retro > users, it could come at the cost of a much more complex spec for how > to reflow along with incompatibilities as clients act differently > when confronted with un-specified situations. At this point, hard-wrapping text for me has a lot of appeal which has *nothing* to do with being retro and is actually about *avoiding* a more complex spec. If text/gemini is hard-wrapped and clients are forbidden from ever reflowing, then people can basically use Markdown syntax (or any other syntax they like!) to produce rich text content without the client needing to actually understand any of that syntax, and hence without the spec needing to describe any of it. It gives you maximum expressive power for minimum spec or implemetation complexity, which is very Gemini. Before you started posting to this list I would have added "and the *only* downside is that it ruins things on narrow phone screens" and then smugly dismissed phones as junk and been happy. But you have covinced me (and I thank you for it!) that this approach has real downsides too, in that it doesn't play as nicely with version control software and is, averaging across the space of all possible editors, actually more difficult for authors to write and to edit than the form where paragraphs are just very long single lines. So, now I recognise this as a compromise solution and I'm not hugely enthusiastic about it. But I really do think that the solution *has* to be a compromise, and that this might be the best possible compromise, all things considered. If anybody really wants to change my mind on this, your time is probably best spent trying to come up with a text/gemini syntax which allows rich text features, degrades gracefully in very simple clients, can be rendered in no more than 100 LOC of Python (and less is better!) and can be succinctly but unambigiously described in the spec. I consider the existence of such a syntax unlikely but not impossible. I'll try to come up with one myself, but unless one materialises that nobody really hates and that can be shown to work well via actual running code by the end of January, I will declare text/gemini hard-wrapped and we can all move on to arguing about the One True Width to wrap at instead. :) Cheers, Solderpunk PS: I'll probably refrain from posting about *this* particular issue on the list over the weekend, otherwise I'll never get anything else done! From solderpunk at SDF.ORG Fri Jan 17 13:33:12 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Fri, 17 Jan 2020 13:33:12 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117111158.GA10236@SDF.ORG> References: <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> Message-ID: <20200117133312.GA7402@SDF.ORG> Hmm, I've just realised something which might salvage this whole mess. It's possible even that what I'm about to describe is exactly what the "long line" folks have been talking about all along without my realising it. Sorry for missing it if so, but I don't think it was ever made explicit! I have always conceptualised our choice as being between two alternatives: 1. Hard-wrapped text which clients display verbatim, line-by-line, exactly the way Gopher works. 2. What I'll call "full blown reflowing", the way HTML and LaTeX work. This involves lines longer than the viewport being split up into multiple shorter lines, but also consecutive non-blank lines shorter than the viewport being joined into fewer, longer lines. Basically, this model of reflow is "paragraph based". Consecutive non-empty lines of text form clumps called paragraphs which are formatted as a whole, whether this results in more or fewer total lines compared to the "source". There is another option that I hadn't thought about until now, which is to do only the first half of 2. above. That is, lines longer than the viewport get broken up nicely at word boundaries into lines of length equal to or less than the viewport width - but that's it. Consecutive shorter lines are *not* joined together. Blank lines in the "source" are rendered, one by one, into empty vertical space. The renderer has no explicit concept of a paragraph. This allows writing things we want to look like paragraphs as individual long lines (easy for most editors, plays nicely with version control) with the knowledge they'll be nicely wrapped to the viewport width, but it doesn't break things like one word per line for emphasis, because the lines won't be sucked up and joined together, and it also doesn't break lists for the same reason (more on lists below). I kind of like this. Unlike the paragraph-oriented web/LaTeX model where ten consecutive newlines and two consecutive newlines are identical, this also allows us to put larger gaps between paragraphs to give the impression of pausing for thought. Neat, huh? This does rescue lists, right? A list of short items: * Mercury * Gemini * Apollo * Shuttle does't get turned into a mess of: * Mercury * Gemini * Apollo Shuttle But multi-line list items get wrapped appropriately. Dumb clients which don't recognise list items at all can do this in an ugly way, by applying exactly the same wrapping code they use for normal lines: * This is a multi-line list item where the beginning of lines after the first one are not offset from the left hand of the viewport to match the offset of the first one. Whereas smarter clients can recognise lines beginning with "*" as being list items and do slightly nicer formatting: * This is a multi-line list item which has been handled by a smarter client which results in a nicer appearance - a strictly optional bit of extra work for ambitious client authors. There doesn't seem to be any need to keep track of whether or not you are inside of a list or not? This syntax is strictly line-based, and recognising a special type of line involves only looking at the first few characters. Recognising Gemini links is of course mandatory, but so long as every other special kind of line can be handled appropriately by just printing it (in the way that list items seemingly can be) then defining special line types for headings, etc. doesn't actually add any extra burden on simple clients. It's basically a question of how many cases you want to handle in a switch statement... This seems pretty great, the only downside is that even bare minimal clients need to be able to wrap long lines to result in readability. I'm not quite as bothered by this as I was before I realised that Python has a standard library function which just does this (it's even pretty smart, e.g. if the final word of a line would make that line longer than the requested width, and the word has a hyphen in it, it'll break at the hyphen if that shortens the lin enough), but it's still not great. I wonder how common that kind of function is across standard libraries? Surely it can't be this simple. I look forward to somebody pointing out the inevitable overlooked fatal flaw... Cheers, Solderpunk From lel at envs.net Fri Jan 17 13:58:04 2020 From: lel at envs.net (lel) Date: Fri, 17 Jan 2020 08:58:04 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117133312.GA7402@SDF.ORG> References: <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> Message-ID: <20200117135804.GA5259@briz> On Fri, Jan 17, 2020 at 01:33:12PM +0000, solderpunk wrote: > It's possible even that what I'm about to describe is exactly what the > "long line" folks have been talking about all along without my > realising it. > ... > There is another option that I hadn't thought about until now, which > is to do only the first half of 2. above. That is, lines longer than > the viewport get broken up nicely at word boundaries into lines of > length equal to or less than the viewport width - but that's it. > Consecutive shorter lines are *not* joined together. Blank lines in > the "source" are rendered, one by one, into empty vertical space. > The renderer has no explicit concept of a paragraph. Yes, this is exactly what I was trying to say. There's no need to join subsequent lines together. That's just reflowing, as far as I can tell. But you don't need to reflow in that specific way unless your content is inconveniently hard-wrapped when you want it to be resizable. This sort of reflowing leads to loss of information if the single line-breaks were intended and meaningful, so it makes more sense to me to not add those line-breaks into the content, but into their rendering on the client-side. This is the happiest I've been to see an email in a while honestly lol ~ lel From solderpunk at SDF.ORG Fri Jan 17 14:27:15 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Fri, 17 Jan 2020 14:27:15 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117135804.GA5259@briz> References: <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> Message-ID: <20200117142715.GA23881@SDF.ORG> On Fri, Jan 17, 2020 at 08:58:04AM -0500, lel wrote: > > Yes, this is exactly what I was trying to say. Ah, right, sorry for not getting that. I think this whole conversation, despite Sean's best efforts to give us formally defined terms, is suffering a lot from the fact that things like "reflowing" mean different things to different people or in different contexts, so we're often talking past each other. We'd still need ``` in this system, right, to avoid e.g. long lines of source getting mangled on very narrow phone screens? So minimal rendering pseudocode looks like this: preformatted = False for line in all_the_lines: if line == "```": preformatted = not preformatted elif preformatted: print(line) elif line.startwith("=>"): handle_link(line) else: wrap_line_to_viewport(line) Where wrap_line_to_viewport may need to be written by the user if there isn't a library function (and should print empty lines if given them). Non-minimal rendering just involves replacing that final "else" clause with more "elifs" to catch e.g. list items or headers (dispatching to different functions, like wrap_bold_line_to_viewport to display headings in bold). You can add as many or as few of those extra clauses as you like to pretty things up, as long as whatever prettiness you want doesn't depend upon anything. If this rendering code *is* fed text/gemini that has been hard wrapped to a width less than or equal to the viewport, that text comes back unmangled (but narrower than it otherwise could be). Are there really no catches beyond this? I'm sure there must be, but if not I guess the next thing to do is to start thinking about how difficult a "good enough" (not necessarily perfect) implementation of wrap_line_to_viewport is in most languages, and decide whether or not we think that burden is too high. Cheers, Solderpunk From lel at envs.net Fri Jan 17 16:07:08 2020 From: lel at envs.net (lel) Date: Fri, 17 Jan 2020 11:07:08 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117142715.GA23881@SDF.ORG> References: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> Message-ID: <20200117160708.GA11117@briz> On Fri, Jan 17, 2020 at 02:27:15PM +0000, solderpunk wrote: > ... > I guess the next thing to do is to start thinking about how > difficult a "good enough" (not necessarily perfect) implementation of > wrap_line_to_viewport is in most languages, and decide whether or not we > think that burden is too high. I don't think it's too high. I just wrote up an awful implementation in the most naive way possible and it came out ~30 lines, with a good deal of that being the edge case of words longer than the viewport width (i handled this by splitting these words with hyphens at the viewport width; i don't know if there are other edge-cases that need to be handled but i can't think of any). I'm attaching it. You can execute it with: ``` python3 wrap.py [length] [text] ``` Or import it and call wrap with first parameter being the text and the second being the viewport width. I can't stress enough how "not perfect" this is (I wrote it up on my phone, of all things, in about 5 minutes, so very little thought was put into making it good) but at the very least it's doable. I know you mention that python can do this natively, but I didn't use any dynamic-typing, and all that's required for this sort of implementation is a method to split by newline and space, so really something like this could be done pretty universally. But it's still not good. ~ lel -------------- next part -------------- A non-text attachment was scrubbed... Name: wrap.py Type: text/x-python Size: 1233 bytes Desc: not available URL: From aaron at ajanse.me Fri Jan 17 16:52:52 2020 From: aaron at ajanse.me (Aaron Janse) Date: Fri, 17 Jan 2020 08:52:52 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117160708.GA11117@briz> References: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> Message-ID: lel said: > Yes, this is exactly what I was trying to say. Ahaha, same here. Thank you, solderpunk, for hearing us out. Sorry for not being so explicit! > then defining special > line types for headings, etc. doesn't actually add any extra burden on > simple clients. It's basically a question of how many cases you want > to handle in a switch statement... This is definitely a big appeal for me. Fancy clients could also wrap quote blocks by putting a greater-than sign on the beginning of each wrapped line, without requiring simple clients to do the same :-) > even bare minimal clients need to be able to wrap long lines to result > in readability. In order to deal with wide clients, yes. But if we did hard wrapping, we'd have to do the exact same thing (but plus reflowing) on narrow clients anyway. > I don't think it's too high. I just wrote up an awful implementation in > the most naive way possible and it came out ~30 lines Plus, most systems have bash, so worst case, someone either calls `fmt`/`par` from the client source code or pipes their client output into `fmt`/`par`. > This is the happiest I've been to see an email in a while honestly lol Definitely. Unless there is a problem I'm not seeing, this sounds very exciting! Cheers! From solderpunk at SDF.ORG Fri Jan 17 17:29:02 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Fri, 17 Jan 2020 17:29:02 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> Message-ID: <20200117172902.GA1432@SDF.ORG> On Fri, Jan 17, 2020 at 08:52:52AM -0800, Aaron Janse wrote: > > Definitely. Unless there is a problem I'm not seeing, this sounds very > exciting! I'm excited by this, too. Because I can't control myself I've already started writing up a rough spec for this, included below for comment. I definitely want to wait until more people have chimed in before getting too excited, and I'd especially like to hear from all, or at least most, of the authors of existing clients before making even a tentative decision. But I have to say, in writing up the below, I get a very good feeling. The description is not exactly short, but it's very *not fiddly* compared to how it would look if we were going to support "reflow in both directions". This "half reflow" approach seems to hit an amazing sweetspot of simplicity, richness and adaptability of display widths. I have figured out how to get vim to let me write this kind of content in a pleasant and easy way, so I think I'm happy to switch. And if anybody *really* hates it, they can still hard-wrap their files and wide clients following these rules will display it properly. Narrow clients will end up with a mess, but that's just the price people will have to pay for sticking to their hard-wrapping guns. Many of them will probably be happy to dismiss mobile clients out of hand, anyway! Feedback on the below very welcome! Cheers, Solderpunk --------- The text/gemini syntax is inspired by and looks visually similar to Markdown, but it is significantly simpler (and consequently less powerful). The syntax is strictly line-based: a file can be processed one line at a time and each line can be handled in a way which does not depend upon any previous or subsequent lines. The only internal state a rendering engine needs to maintain is a single boolean variable (for preformatting mode, described below). The lines of a text/gemini file come in 8 different types. It is possible to unambiguously determine the type of a line by only considering its first three characters. Knowing a line's type is the only information necessary to know how to handle it correctly. Just like Gemini's status code system is designed so that simple clients can ignore the second digit of the code and still function correctly, the text/gemini syntax is designed so that simple clients can treat several different types of line identically and still provide a usable representation of the document. A bare-minimum client need only recognise four different types of line. These are: 1. PREFORMATTED TOGGLE LINES Lines consisting of only and exactly three back ticks (```) are preformatted toggle lines. These lines should not be displayed to the user and are instead used to toggle preformatted mode on and off (the mode is off when the parser is intialised). When preformatted mode is on, the usual rules for identifying line types are suspended and all lines should be unconditionally identified as PREFORMATTED TEXT LINES. 2. PREFORMATTED TEXT LINES Preformatted text lines should be presented to the user in a monowidth font exactly as they appear in the text/gemini file. Lines longer than the client's viewport must not be wrapped, leading or trailing whitespace must not be removed, etc. Handling of lines longer than the viewport is client-specific. Advanced clients may display a horizontal scrollbar. Simple clients may simply truncate the line. 3. LINK LINES Lines beginning with the two characters => are link lines. We all know how these work by now. 4. TEXT LINES All other lines are TEXT LINES. They should be presented to the user in a client-specific "pleasing manner". Lines longer than the client's viewport should be wrapped into multiple lines of a suitable length. Variable width fonts may be used. Blank lines are a special case of TEXT LINES and should be reproduced in the output. It is important to realise that while Markdown, HTML, LaTeX and many other document markup formats are "block based" or "paragraph based", the text/gemini format is not. Consecutive, non-blank lines of text which are much shorter than the client's viewport should *not* be combined into longer lines. Equivalently, *every* newline character in a text/gemini file is significant, not only consecutive pairs of newline characters. Clients ignoring this fact will produce incorrect output. Advanced clients may also recognise the following additional line types. Simple clients may treat all line types below as TEXT LINES with no loss of essential function. 5. HEADING LINES Lines beginning with one, two or three consecutive # characters are HEADING LINES, corresponding to headings, subheadings or susubheadings respectively. The text following the # characters (with leading whitespace removed) constitute the heading and should be displayed to the user. Clients MAY choose to apply special styling to headings to distinguish them from ordinary lines. However, the primary purpose of HEADING LINES is to represent the internal structure of the document in a machine-readable way. Advanced clients can use this information to, e.g. display a hierarchically formatted "table of contents" for a long document in a side-pane, allowing users to easily jump to specific sections without excessive scrolling. Or CMS-style tools automatically generating menus or Atom/RSS feeds for a directory of text/gemini files can use the first heading in a file as a human-friendly label for links to it. 6. UNORDERED LIST ITEMS Lines beginning with a * are UNORDERED LIST ITEMS. This line type exists purely for stylistic reasons. The * may be replaced in advanced clients by a bullet symbol. Any text after the * character should be presented to the user as if it were a TEXT LINE, i.e. wrapped to fit the viewport. Advanced clients can take the space of the bullet symbol into account when performing wrapping and ensure that all lines of text corresponding to the item are aligned with one another. 7. ORDERED LIST ITEMS As above with obvious changes. From aaron at ajanse.me Fri Jan 17 17:59:18 2020 From: aaron at ajanse.me (Aaron Janse) Date: Fri, 17 Jan 2020 09:59:18 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117172902.GA1432@SDF.ORG> References: <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> Message-ID: <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> > Lines consisting of only and exactly three back ticks (```) are preformatted > toggle lines. Hmm, this might confuse some people who are used to markdown's allowance of specifying the language: ```python print("Hello!") ``` Plus, if we *allow* people to specify the language, maybe some clients could implement syntax highlighting. Would that be too complex? If not, what would happen if people add text to the closing three ticks? > Lines longer than the client's viewport must not be wrapped, leading or > trailing whitespace must not be removed, etc. Handling of lines longer than > the viewport is client-specific These two sentences contradict each other, I think. Maybe state that clients MUST NOT remove trailing space, etc, but state that clients SHOULD allow readers to copy the text such that it can be pasted directly into a text editor then run (this would allow the 100-line python client to just print the lines; terminals should un-soft-wrap lines longer than the viewport when copied to the clipboard). > They should be presented to the user in a client-specific "pleasing manner" > Variable width fonts may be used. I *love* that this is part of the spec! > 6. UNORDERED LIST ITEMS Are minus signs allowed for unordered list items? How about plus signs etc? I'm somewhat in favor of limiting bullets to astrisks and minuses. Can top-level bullets begin with a space? What about nested bullet points? > 7. ORDERED LIST ITEMS I'd further specify what is/isn't allowed here. For example: - Some people may try `1)` instead of `1.`. I personally think that spec should say that only the latter is allowed but clients may choose to *render* ordered lists as the former - What about nested ordered bullet points. Wouldn't they start with whitespace? - Markdown allows lists to be auto-numbered. For example, the nested bullet points would be re-numbered from 1 to 5 in markdown: 1. one 1. two 5. three 2. four 1. five - What about lettered lists (A-Z)? I think these would be cool to have but I doubt they'd be worth the complexity. I'd be explicit in the spec - What about roman numerals? I don't think these should be allowed While we could leave some stuff un-specified and see how things play out, I think there could be some value in limiting authors for the sake of preventing a scenario where making a client is so complex that very few nice clients exist (as we see in the world of web browsers). > The lines of a text/gemini file come in 8 different types. Hmm, I only saw 7 different types specified. I also recommend you specify that people may use greater-than-symbol quotes, which may be nested. I'd recommend that authors MUST NOT unnecessarily hard-wrap their quotes, suggesting that advanced clients MAY add a visual greater-than symbol to the beginning of each wrapped line. Example: > hello this is wider than the viewport Displayed by advanced clients as: > hello this is > wider that the > viewport --- I asked a ton of questions, but I still like the direction we're going. I'd just like to bring up one more thing: What about Gemini proxies of comment thread sites, such as hacker news (*waves to Michael Lazar's awesome HN Gopher proxy*). I think we should take those into consideration. If we wanted to introduce new syntax (which is a bit crazy, but fun), we could use pipes like greater-than signs. We could use the exact same code for this that we would use for fancy-wrapping greater-than-sign quotes. For example, the source code would be: | # John | this is a comment wider than the very narrow viewport | | | # Joe | | and this is a sub comment tthat is very very very long! | | | | | # You | | | and this is a sub sub comment! | | | # Bob | | and this is a sub comment The output on a narrow viewport would be: | # John | this is a comment wider | than the very narrow | viewport | | | # Joe | | and this is a sub | | comment that is very | | very very long! | | | | | # You | | | and this is a sub | | | sub comment! | | | # Bob | | and this is a sub | | comment I know it's super crazy, but it sounds simple to implement. I was considering mentioning it in its own thread, but I think this is the most relevant time to bring it up. Cheers! From sean at conman.org Sat Jan 18 02:16:27 2020 From: sean at conman.org (Sean Conner) Date: Fri, 17 Jan 2020 21:16:27 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117172902.GA1432@SDF.ORG> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> Message-ID: <20200118021627.GF3354@brevard.conman.org> It was thus said that the Great solderpunk once stated: > > Feedback on the below very welcome! Red alert! Raise shields! Strap in! This is going to be a bumpy ride. Attached is a sample document that I created that I'm trying to format per the spec below. I'm already running into trouble, else I would have not replied with this particular response. > The text/gemini syntax is inspired by and looks visually similar to > Markdown, but it is significantly simpler (and consequently less > powerful). The syntax is strictly line-based: a file can be processed > one line at a time and each line can be handled in a way which does > not depend upon any previous or subsequent lines. The only internal > state a rendering engine needs to maintain is a single boolean > variable (for preformatting mode, described below). > > The lines of a text/gemini file come in 8 different types. I only see 7 listed below. > It is > possible to unambiguously determine the type of a line by only > considering its first three characters. Knowing a line's type is the > only information necessary to know how to handle it correctly. Not quite true, even according to this document. Leading white space in lists is the glaring exception here. > Just like Gemini's status code system is designed so that simple > clients can ignore the second digit of the code and still function > correctly, the text/gemini syntax is designed so that simple clients > can treat several different types of line identically and still > provide a usable representation of the document. A bare-minimum > client need only recognise four different types of line. These are: Yet seven are listed. > 1. PREFORMATTED TOGGLE LINES > > Lines consisting of only and exactly three back ticks (```) are > preformatted toggle lines. These lines should not be displayed to the > user and are instead used to toggle preformatted mode on and off (the > mode is off when the parser is intialised). When preformatted mode is > on, the usual rules for identifying line types are suspended and all > lines should be unconditionally identified as PREFORMATTED TEXT LINES. > > 2. PREFORMATTED TEXT LINES > > Preformatted text lines should be presented to the user in a monowidth > font exactly as they appear in the text/gemini file. Lines longer > than the client's viewport must not be wrapped, leading or trailing > whitespace must not be removed, etc. Handling of lines longer than the > viewport is client-specific. Advanced clients may display a > horizontal scrollbar. Simple clients may simply truncate the line. No real problems so far. > 3. LINK LINES > > Lines beginning with the two characters => are link lines. We all know > how these work by now. Again, no real problem. > 4. TEXT LINES > > All other lines are TEXT LINES. They should be presented to the user > in a client-specific "pleasing manner". Lines longer than the > client's viewport should be wrapped into multiple lines of a suitable > length. Variable width fonts may be used. Blank lines are a special > case of TEXT LINES and should be reproduced in the output. > > It is important to realise that while Markdown, HTML, LaTeX and many > other document markup formats are "block based" or "paragraph based", > the text/gemini format is not. Consecutive, non-blank lines of text > which are much shorter than the client's viewport should *not* be > combined into longer lines. Equivalently, *every* newline character > in a text/gemini file is significant, not only consecutive pairs of > newline characters. Clients ignoring this fact will produce incorrect > output. Fair enough. And so far, things are okay. > Advanced clients may also recognise the following additional line > types. Simple clients may treat all line types below as TEXT LINES > with no loss of essential function. It's here we start running into trouble. > 5. HEADING LINES > > Lines beginning with one, two or three consecutive # characters are > HEADING LINES, corresponding to headings, subheadings or susubheadings > respectively. The text following the # characters (with leading > whitespace removed) The parenthetical here is ambiguous. Does it refer to this issue? #A title ## A title with space between the '#' and text ### Even more white space or #A title ##A title with leading space before the '#' ###Even more white space I'm thinking the former now that I'm replying, but my code deals with both cases combined, so I can handle: # A title ## A title with spaces ### Yippee! Spaces galore! Almost---tabs (and yes, I do use tabs---I like me the tabs) are an issue and while I can handle them (I have code that will expand them up to 8 spaces) not everybody has code to deal with this. So question: WHAT ABOUT TABS? They WILL show up. > constitute the heading and should be displayed to > the user. Clients MAY choose to apply special styling to headings to > distinguish them from ordinary lines. However, the primary purpose of > HEADING LINES is to represent the internal structure of the document > in a machine-readable way. Advanced clients can use this information > to, e.g. display a hierarchically formatted "table of contents" for a > long document in a side-pane, allowing users to easily jump to > specific sections without excessive scrolling. Or CMS-style tools > automatically generating menus or Atom/RSS feeds for a directory of > text/gemini files can use the first heading in a file as a > human-friendly label for links to it. So another question. I have some headings like this: ### Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. Given a screen width of 40, which of the four below should be displayed? ### Lorem ipsum dolor sit amet, consect ### Lorem ipsum dolor sit amet, cons... ### Lorem ipsum dolor sit amet, ### consectetur adipiscing elit. Cras ### sodales eget nisi quis condimentum. ### Donec ipsum arcu, fermentum eu ### ullamcorper sit amet, facilisis id ### nunc. Class aptent taciti sociosqu ### ad litora torquent per conubia ### nostra, per inceptos himenaeos. Nam ### tempus nulla ut dolor luctus ### malesuada. Suspendisse orci sem, ### semper at maximus non, pharetra et ### justo. Quisque lectus arcu, viverra ### ac convallis eu, vulputate ut enim. ### Nulla aliquam, lacus consequat ### suscipit facilisis, nisl tortor ### facilisis nisi, vel mattis eros ### arcu sed tellus. Duis quis lectus ### pellentesque, posuere dolor ut, ### sodales massa. Proin vel blandit ### mauris. ### Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. ### Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. Or is this way into the "you have *got* to be kidding!" territory? I swear, I'm not trying to take these things to the extreme ... well, okay, I *am* trying to take these things to the exteme, but only to find out where the borders are. > 6. UNORDERED LIST ITEMS > > Lines beginning with a * are UNORDERED LIST ITEMS. This line type > exists purely for stylistic reasons. The * may be replaced in > advanced clients by a bullet symbol. Any text after the * character > should be presented to the user as if it were a TEXT LINE, i.e. > wrapped to fit the viewport. Advanced clients can take the space of > the bullet symbol into account when performing wrapping and ensure > that all lines of text corresponding to the item are aligned with one > another. Okay, several questions here. First off, replace the '###' with '*' in the above example---how to properly format a list item that is ridiculously long. Second question---this list: * lorem * ipsum * dolor * Fusce faucibus ... (I'll save you the text) * Nunc pharetra ... * consectetur * adipiscing * elit * Cras * sodales * eget * nisi This is a nested list, so the lines start with a blank line. This list indents one space, but it could be two spaces per index, three, eight. Also, this list: * lorem * ipsum * dolor * Fusce faucibus ... * Nunc pharetra ... * consectetur * adipiscing * elit * Cras * sodales * eget * nisi Uses tabs (ah, lovely tabs). Are indents preserved? Transformed? What if there's mixed spaces and tabs (one line is a tab and two space, another line is 10 spaces---with a default tab stop of 8, they both line up)? > 7. ORDERED LIST ITEMS > > As above with obvious changes. Not so fast there! Again, the same issues as above, but now here are some ordered lists that are obvious *to me* but maybe not to others: 1 lorem 1 ipsum 1 dolor 1 Fusce faucibus ... 2 Nunc pharetra ... 2 consectetur 3 adipiscing 4 elit 2 Cras 3 sodales 2 eget 3 nisi The following is how I learned outlining in school: 1 lorem A ipsum i dolor a Fusce faucibus ... b Nunc pharetra ... ii consectetur iii adipiscing iv elit B Cras C sodales 2 eget 3 nisi And in case you don't want to support Roman numerals: 1 lorem A ipsum 1 dolor A Fusce faucibus ... B Nunc pharetra ... 2 consectetur 3 adipiscing 4 elit B Cras C sodales 2 eget 3 nisi And again, all the same questions apply from unordered lists. That's it for now. I'm going to hold off on my implementation until some of this is cleared up, but please do check out the attached document as a warning of what might be possible with this format. -spc (Will torture specs for food ...) -------------- next part -------------- # Sample Gemini Text File ## Very long lines, blank lines between them Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. Aenean vehicula eu eros vel feugiat. Quisque sagittis metus eu nisl dapibus condimentum. Aenean ipsum justo, sagittis vel ipsum sit amet, fermentum convallis elit. Ut congue scelerisque velit, nec euismod nulla gravida quis. Duis faucibus tempus ligula, non malesuada neque lobortis quis. Nam neque magna, ornare eu dui ut, porttitor tincidunt purus. Duis id malesuada ante. Suspendisse gravida condimentum nisl, eget gravida dui pellentesque et. Fusce tempor leo nulla, non posuere sem maximus eget. Integer non maximus quam. Nam ac felis ut elit aliquam aliquam. Curabitur laoreet metus nulla, a ornare lorem molestie a. Sed id libero vel nunc lobortis lacinia sed quis metus. Sed feugiat eget ipsum et commodo. Fusce condimentum est ut arcu imperdiet, vel porta felis tincidunt. Aliquam quis molestie libero, sit amet luctus quam. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Suspendisse potenti. Integer quis eros neque. Sed vulputate condimentum est et cursus. Nunc urna ante, euismod quis tempus ac, aliquam at felis. Sed hendrerit ex eu odio sodales fermentum. Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. ## Very long lines, one after the other Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. Aenean vehicula eu eros vel feugiat. Quisque sagittis metus eu nisl dapibus condimentum. Aenean ipsum justo, sagittis vel ipsum sit amet, fermentum convallis elit. Ut congue scelerisque velit, nec euismod nulla gravida quis. Duis faucibus tempus ligula, non malesuada neque lobortis quis. Nam neque magna, ornare eu dui ut, porttitor tincidunt purus. Duis id malesuada ante. Suspendisse gravida condimentum nisl, eget gravida dui pellentesque et. Fusce tempor leo nulla, non posuere sem maximus eget. Integer non maximus quam. Nam ac felis ut elit aliquam aliquam. Curabitur laoreet metus nulla, a ornare lorem molestie a. Sed id libero vel nunc lobortis lacinia sed quis metus. Sed feugiat eget ipsum et commodo. Fusce condimentum est ut arcu imperdiet, vel porta felis tincidunt. Aliquam quis molestie libero, sit amet luctus quam. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Suspendisse potenti. Integer quis eros neque. Sed vulputate condimentum est et cursus. Nunc urna ante, euismod quis tempus ac, aliquam at felis. Sed hendrerit ex eu odio sodales fermentum. Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. ## Very short lines (how I tend to blog) Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. ## Links => gemini://example.net/ Lorem ipsum dolor sit amet => gemini://example.net/ Class aptent taciti sociosqu ad litora torquent per conubia nostra => gemini://example.net/ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. => gemini://example.net/ => gemini://example.net/lorem/ipsum/dolor/sit/amet/consectetur/adipiscing/elit/cras/sodales/eget/nisi/quis/condimentum/donec/ipsum/arcu/fermentum/eu/ullamcorper/sit/amet/facilisis/id/nunc/class/aptent/taciti/sociosqu/ad/litora/torquent/per/conubia/nostra/per/inceptos/himenaeos/nam/tempus/nulla/ut/dolor/luctus/ ## Preformatted Text Lines ``` quis x1 = facilisis "lpeg" quis x2 = "..." quis x3 = x1.x4(x1.x5("AZ","az")^1) + (x1.x6(1) - x1.x5("AZ","ax"))^1 quis x7 = x1.x8(x3^1) quis x9 = x7:lobortis(x2) quis x10 = 1 -- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales -- eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper -- sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora -- torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut -- dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, -- pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, -- vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, -- nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis -- lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit -- mauris. elit x11 nisi sed.liguia() dolor eget x11 ~= "" eu sed.aliquam:nunc(lectus.amet(' [%q]=%q,\n',x11,x9[i]:lobortis())) x10 = x10 + 1 sit sed.aliquam:nunc(x11,'\n') consectetur consectetur ``` ## Lists ### Bullet List 1 * lorem * ipsum * dolor * Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. * Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. * consectetur * adipiscing * elit * Cras * sodales * eget * nisi ### Bullet List 2 * lorem * ipsum * dolor * Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. * Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. * consectetur * adipiscing * elit * Cras * sodales * eget * nisi ### Ordered List 1 1 lorem 1 ipsum 1 dolor 1 Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. 2 Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. 2 consectetur 3 adipiscing 4 elit 2 Cras 3 sodales 2 eget 3 nisi ### Ordered List 2 1 lorem A ipsum i dolor a Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. b Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. ii consectetur iii adipiscing iv elit B Cras C sodales 2 eget 3 nisi ### Ordered List 3 1 lorem A ipsum 1 dolor A Fusce faucibus dui et consectetur aliquet. Nullam augue magna, bibendum sit amet commodo at, sagittis non leo. Sed eget mauris eget arcu vulputate vulputate. Morbi non gravida dolor, in mollis turpis. Ut quis tempor elit. Aenean nec arcu vitae justo gravida placerat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vel suscipit nisi. Curabitur vitae elit ultricies, ultrices dolor sed, vehicula augue. Phasellus euismod ligula sit amet mi volutpat tempor. Nunc scelerisque mauris dui, sed placerat nibh tempus non. Mauris eleifend purus nec tincidunt dapibus. Maecenas tincidunt volutpat varius. Phasellus eu congue risus. Nam venenatis erat non auctor ullamcorper. B Nunc pharetra sem nec velit tempus, sed malesuada felis accumsan. Vestibulum egestas ex nisl, sit amet rhoncus mauris laoreet at. Ut neque lorem, tempus et dictum non, laoreet ac mauris. Maecenas consectetur blandit neque eget maximus. Duis tincidunt elementum lorem, at varius nisl dapibus vel. In blandit ipsum sed molestie commodo. Quisque aliquet nunc eget pretium viverra. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Morbi ligula nisi, sollicitudin tincidunt efficitur nec, lobortis eu diam. Vestibulum convallis erat velit, semper semper augue sodales in. 2 consectetur 3 adipiscing 4 elit B Cras C sodales 2 eget 3 nisi ## Long Titles ### Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. ### Aenean vehicula eu eros vel feugiat. Quisque sagittis metus eu nisl dapibus condimentum. Aenean ipsum justo, sagittis vel ipsum sit amet, fermentum convallis elit. Ut congue scelerisque velit, nec euismod nulla gravida quis. Duis faucibus tempus ligula, non malesuada neque lobortis quis. Nam neque magna, ornare eu dui ut, porttitor tincidunt purus. Duis id malesuada ante. Suspendisse gravida condimentum nisl, eget gravida dui pellentesque et. And this should provide for some long discussions. I hope. From sean at conman.org Sat Jan 18 03:47:21 2020 From: sean at conman.org (Sean Conner) Date: Fri, 17 Jan 2020 22:47:21 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200117172902.GA1432@SDF.ORG> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> Message-ID: <20200118034720.GG3354@brevard.conman.org> It was thus said that the Great solderpunk once stated: > > Feedback on the below very welcome! Okay, I went ahead and implemented some of this spec. I didn't bother with the list stuff as that has to be clarified. But the rest I did, and so far, the implementation wasn't *too* bad---about 50 lines, but I'm using a library to wrap text which may or may not exist in other languages. Anyway, you guys can test out a server rendered version at various widths (you can even specify the width) at: gemini://gemini.conman.org/test/testreflow.gemini Some notes---the link section may or may not be how clients render links---since the output is pure text, I decided to just make them look distinct. Also, that large block of text (with the header "Very long lines, one after the other") I might change (add a blank line that doesn't exist) but that would complicate the implementation (in case there *is* a blank line in the input), so I'm still thinking on that one. But in any case, here's a sample implementation (the link to the code is in the page mentioned above) that I'm presenting for your bikeshedding excitement. -spc (Remember---I haven't implemented lists yet ... ) From lazar.michael22 at gmail.com Sat Jan 18 05:02:42 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Sat, 18 Jan 2020 00:02:42 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200116221202.GC23953@SDF.ORG> References: <20200115202137.GA21763@SDF.ORG> <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <546c2284-4b16-4ec1-8a9f-a072e0773799@www.fastmail.com> <20200116221202.GC23953@SDF.ORG> Message-ID: On Thu, Jan 16, 2020 at 5:12 PM solderpunk wrote: > > On Thu, Jan 16, 2020 at 01:09:18PM -0800, Aaron Janse wrote: > > Oops, I meant to mention two things: > > 1. Clients are already using TLS. I think wrapping is significantly > > easier than encryption. > > I was about to quip that the TLS is made really easy by high-level > library support while the wrapping has to be done manually, but then I > thought to check and, holy heck, Python has a `textwrap` module I wasn't > aware of. Python's textwrap module is fundamentally flawed for unicode and they have no intention of ever fixing it [0]. Once you start going down the rabbit hole of CJK characters, emojis, grapheme clusters, etc. it becomes exceedingly hard to figure out how to correctly determine the width of unicode text. You can get it working 99% of the time, but there's always those fringe cases that no one thinks about until somebody files a bug report. I don't know if this has any bearing on the discussion, but it's worth keeping in the back of your mind if you intend to make unicode a first-class citizen. I would be cautious about calling text wrapping "significantly" easier than TLS or anything else for that matter. This was actually one of the things that drew me to gopher in the first place, I could assume everything was ASCII and throw all of that complexity out the window. [0] https://bugs.python.org/issue24665 From solderpunk at SDF.ORG Sat Jan 18 11:45:03 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 11:45:03 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> Message-ID: <20200118114503.GA4518@SDF.ORG> On Fri, Jan 17, 2020 at 09:59:18AM -0800, Aaron Janse wrote: > > Lines consisting of only and exactly three back ticks (```) are preformatted > > toggle lines. > > Hmm, this might confuse some people who are used to markdown's allowance of > specifying the language: > > ```python > print("Hello!") > ``` > > Plus, if we *allow* people to specify the language, maybe some clients could > implement syntax highlighting. Would that be too complex? Ah, yes, Tomasino mentioned this earlier. I guess it is harmless (?) to change the definition of these lines to any which begin with ```, not which consist strictly of ```. That lets sufficiently fancy clients support this, the rest can just ignore it. These lines are never actually shown to the user, so it doesn't matter what junk comes after the ```. > If not, what would happen if people add text to the closing three ticks? Well, that's unambiguous. If we didn't make the above change, lines with text after the three ticks would not meet the definition of a preformatted toggle line, so a client would identify them as a text line and they would be rendered accordingly. > > Lines longer than the client's viewport must not be wrapped, leading or > > trailing whitespace must not be removed, etc. Handling of lines longer than > > the viewport is client-specific > > These two sentences contradict each other, I think. Hmm, okay, I guess they technically do. Clients can handle over-long lines however they want as long as they don't wrap them? > Are minus signs allowed for unordered list items? How about plus signs etc? > I'm somewhat in favor of limiting bullets to astrisks and minuses. I'm fairly strongly in favour of limiting everything to exactly one way of doing it. The other day I skimmed the CommonMark spec https://spec.commonmark.org/0.29/ to reassure myself we weren't doing too much wheel re-invention. Holy heck, there are so many different ways to do everything! Simple is best, less is more, one is enough! > Can top-level bullets begin with a space? What about nested bullet points? If a line begins with a space, then it doesn't begin with a *! So then it's by definition not an UNORDERED LIST ITEM. It's a TEXT LINE. I really did mean "begins with" everywhere I said it. IMHO this syntax quickly becomes really unappealing unless the task of deciding which type of line a line is remains dirt simple. Yes, this means no nested lists. It may seem like I've gone nuts and suddenly happily let a whole bunch of complicated stuff into the spec, but I really haven't! This is still supposed to be a very simple syntax, which inevitably comes with limitations. There is no nested anything in the whole syntax. Reliable detection of nestedness in the face of even slight variation in how authors write things will require considering lines in the context of previous or subsequent lines, and that's a no-no here. I'm only willing to allow all these fun toys in if we can do it in such a way that an adequate rendering job can be done by considering each line of the file in perfect isolation, with a single pass of the entire document. > > 7. ORDERED LIST ITEMS > > I'd further specify what is/isn't allowed here. For example: > - Some people may try `1)` instead of `1.`. I personally think that spec should > say that only the latter is allowed but clients may choose to *render* > ordered lists as the former > - What about nested ordered bullet points. Wouldn't they start with whitespace? > - Markdown allows lists to be auto-numbered. For example, the nested bullet > points would be re-numbered from 1 to 5 in markdown: > 1. one > 1. two > 5. three > 2. four > 1. five > - What about lettered lists (A-Z)? I think these would be cool to have but I > doubt they'd be worth the complexity. I'd be explicit in the spec > - What about roman numerals? I don't think these should be allowed > > While we could leave some stuff un-specified and see how things play out, I > think there could be some value in limiting authors for the sake of preventing > a scenario where making a client is so complex that very few nice clients exist > (as we see in the world of web browsers). Okay, I totally goofed up here in declaring the changes as "obvious". I meant "obvious to everybody who has read gemini://mozz.us/markdown/design_document and accepted is as their personal Lord and saviour". Well, actually, I read that and foolishly assumed the nice approach to ordered list items was more or less the same as standard Markdown. Turns out it's not and Michael has done very good simplifying work here. As said above, I am strongly in favour of there being exactly one way to do things, and of identifying a line's type being brutally simple. This totally rules out letting authors actually write numbers. Actually having a number followed by a period define such a line type would also bring a very high risk of falsely identifying ordered list item lines when processing hard-wrapped text if a sentence ending in a number, like "Ten Gemini crews flew low Earth orbit (LEO) missions during 1965 and 1966." was wrapped in such a way that the final word was at the start of a line. Yes, I know, this new syntax works best when there is no hard wrapping, so that we can use lines to determine the scope of certain kinds of "specialness", but I will be VERY HAPPY if we can make the syntax robust enough that it can still be applied to rare instances of hard-wrapped content without much going wrong. So, lines beginning with a + (not any whitespace, but a +!) are ORDERED LIST ITEM lines. Clients who want to be fancy can add a little bit of extra internal state to their rendering code and can replace the +s with incrementing numbers. It's the client's choice whether it uses 1. 2. 3. or 1) 2) 3) or i> ii> iii> or whatever else. Very fancy clients can let the user decide. Yes, this means content authors lose precise control over how their content is rendered (while retaining precise control over the *semantics* of their content, i.e. authors decide whether an item is ordered or not). I'm not just okay with this, I'm actively happy about it. The web paradigm where readers are subordinate to authors with regards to layout is a cause of many different kinds of grief. Good riddace to it! > Hmm, I only saw 7 different types specified. Okay, turns out I can't count in a hurry. :) I actually just wrote N when I first wrote that sentence, then when I ran out of time to sketch this thing out I went back, did a quick, incorrect count, and changed it. In the actual spec I'll double check, and the number will be whatever it is - I didn't mean for the list I sent out to be exhaustive, although I also think we should resist the urge to add every nice little thing we can think of. It'd be great if we kept the total number to 10 or less. > I also recommend you specify that > people may use greater-than-symbol quotes, which may be nested. I have no problem with quotes, but I'm not thrilled by the nesting prospect. > that authors MUST NOT unnecessarily hard-wrap their quotes, suggesting that > advanced clients MAY add a visual greater-than symbol to the beginning of each > wrapped line. > > Example: > > > hello this is wider than the viewport > > Displayed by advanced clients as: > > > hello this is > > wider that the > > viewport This is exactly how I'd expect advanced clients to handle this, and I think this whole idea is implicit in the design of this syntax: the start of a line indicates what kind of line it is, and the scope of that type is precisely that line. A hard-wrapped quote with a > at the start of each line is, in this syntax, actually several distinct consecutive quotes. > What about Gemini proxies of comment thread sites, such as hacker news (*waves > to Michael Lazar's awesome HN Gopher proxy*). I think we should take those > into consideration. Hmm. I would want to think a bit before I lay down a hard statement on this because I don't want to impose too much of my own ideology on Gemini, as it's supposed to be a general-purpose tool....but I am not excited by verbatim dragging of mainstream web 2.0 cultural concepts like comment threads into Gemini. The circle of Gemini early-adopters overlaps considerably with the "Small Internet" / "Slow Internet" movement, which I guess has coloured how I think about the protocol. Thus the idea of adding something into the spec specifically to support visualising deep comment threads in the web-conventional way kind of gives me the heebie-jeebies. Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 12:59:16 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 12:59:16 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200115221117.GB21763@SDF.ORG> <644e55a3-2da6-4c54-b2de-dd435058e2bd@www.fastmail.com> <20200116151001.GA24028@SDF.ORG> <359fadc4-9c53-4700-abc5-cc132c107c9a@www.fastmail.com> <546c2284-4b16-4ec1-8a9f-a072e0773799@www.fastmail.com> <20200116221202.GC23953@SDF.ORG> Message-ID: <20200118125916.GB4518@SDF.ORG> On Sat, Jan 18, 2020 at 12:02:42AM -0500, Michael Lazar wrote: > Python's textwrap module is fundamentally flawed for unicode and they have no > intention of ever fixing it [0]. Once you start going down the rabbit hole of > CJK characters, emojis, grapheme clusters, etc. it becomes exceedingly hard > to figure out how to correctly determine the width of unicode text. You can > get it working 99% of the time, but there's always those fringe cases that > no one thinks about until somebody files a bug report. ... God, I hate computers. But, many thanks for bringing this to my attention. > I don't know if this has any bearing on the discussion, but it's worth keeping > in the back of your mind if you intend to make unicode a first-class citizen. Unicode is already a first-class citizen in Gemini (text/gemini is assumed to be UTF-8 if a different encoding is not explicitly provided in the response header), and I don't think I have any interest in changing that. As for the present discussion...well, it's obvious this problem is no less of a problem under paragraph-oriented "bidirectional" reflowing. It's not obvious to me if it's less of a problem under a Gopher-style hard-wrapping to a pre-defined maximum width model....I suppose if the width of line including CJK characters is dependent upon the combination of font and terminal being used (I don't know if it is, but it seems probable) then it's not actually possible for a CJK-using author to comply with a spec like "Hard-wrap all your content at X characters"... Hmm... Solderpunk From tomasino at lavabit.com Sat Jan 18 13:10:39 2020 From: tomasino at lavabit.com (James Tomasino) Date: Sat, 18 Jan 2020 13:10:39 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118114503.GA4518@SDF.ORG> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> Message-ID: <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> From: solderpunk > Yes, this means no nested lists. It may seem like I've gone nuts and > suddenly happily let a whole bunch of complicated stuff into the spec, > but I really haven't! This is still supposed to be a very simple > syntax, which inevitably comes with limitations. There is no nested > anything in the whole syntax. Love the + thing. If you think of lists and sublists hierarchically rather than nested then we do have a parallel, the # headers! Ordered list: + List item one + List item 2 ++ Level 2 item 1 ++ Level 2 item 2 Unordered list: * List 1 ** Deeper *** Maximum deepitude And it maintains the whole "read the first 3 characters to determine what this is" rule. No extra tabbing or spacing to get in the way and follows similar conventions to the headings. From solderpunk at SDF.ORG Sat Jan 18 14:11:44 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 14:11:44 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> References: <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> Message-ID: <20200118141144.GC4518@SDF.ORG> On Sat, Jan 18, 2020 at 01:10:39PM +0000, James Tomasino wrote: > From: solderpunk > > Love the + thing. If you think of lists and sublists hierarchically rather than nested then we do have a parallel, the # headers! > > ... > > And it maintains the whole "read the first 3 characters to determine what this is" rule. No extra tabbing or spacing to get in the way and follows similar conventions to the headings. > Ah, now *that*'s nice! Having a consistent system makes the whole thing easier to learn and remember. Good thinking! Cheers, Solderpunk From dave at ratfactor.com Sat Jan 18 14:46:47 2020 From: dave at ratfactor.com (Dave Gauer) Date: Sat, 18 Jan 2020 09:46:47 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> Message-ID: On 1/18/20 8:10 AM, James Tomasino wrote: > From: solderpunk > >> Yes, this means no nested lists. It may seem like I've gone nuts and >> suddenly happily let a whole bunch of complicated stuff into the spec, >> but I really haven't! This is still supposed to be a very simple >> syntax, which inevitably comes with limitations. There is no nested >> anything in the whole syntax. > > > Love the + thing. If you think of lists and sublists hierarchically rather than nested then we do have a parallel, the # headers! > > Ordered list: > + List item one > + List item 2 > ++ Level 2 item 1 > ++ Level 2 item 2 > > Unordered list: > * List 1 > ** Deeper > *** Maximum deepitude > > And it maintains the whole "read the first 3 characters to determine what this is" rule. No extra tabbing or spacing to get in the way and follows similar conventions to the headings. Hello! I've been lurking throughout the entire long history of this discussion. ("Wanted to contribute, but other commitments...etc. etc.") I just wanted to pipe in to share my excitement about this simple 3-char rule. I *love* the simplicity of "context-free" line-based parsing. This reminds me of several times in which I've used an "indent level + sort order" in databases to simulate complex nested tree hierarchies to make rendering to the screen absolutely trivial. For some reason, it had never occurred to me that counting the number of stars '*' at the beginning of the line logically amounted to the same thing. This is wonderful. I'm also strongly in favor of the Maximum Deepitude rule. <3 -ratfactor From julien at typed-hole.org Sat Jan 18 15:21:04 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 16:21:04 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> Message-ID: On 1/18/20 2:10 PM, James Tomasino wrote: > Ordered list: > + List item one > + List item 2 > ++ Level 2 item 1 > ++ Level 2 item 2 > > Unordered list: > * List 1 > ** Deeper > *** Maximum deepitude > > And it maintains the whole "read the first 3 characters to determine what this is" rule. No extra tabbing or spacing to get in the way and follows similar conventions to the headings. I really like this idea combined with the "read the first 3 characters", simple and elegant! From solderpunk at SDF.ORG Sat Jan 18 15:39:47 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 15:39:47 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> Message-ID: <20200118153947.GD4518@SDF.ORG> On Sat, Jan 18, 2020 at 04:21:04PM +0100, Julien Blanchard wrote: > I really like this idea combined with the "read the first 3 characters", > simple and elegant! Julien, as the author of multiple Gemini clients, how do you feel about the prospect of most text/gemini content having very long lines which need to be wrapped to fit the viewport? Cheers, Solderpunk From julien at typed-hole.org Sat Jan 18 16:41:05 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 17:41:05 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118153947.GD4518@SDF.ORG> References: <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118153947.GD4518@SDF.ORG> Message-ID: On 1/18/20 4:39 PM, solderpunk wrote: > Julien, as the author of multiple Gemini clients, how do you feel about > the prospect of most text/gemini content having very long lines which > need to be wrapped to fit the viewport? Both my clients try to display content as the author wrote it and I parse it line by line so I don't see any issue with the proposal as long as I can get the viewport size in my clients. Both NCurses and GTK should provide that I'm quite sure. Identifying what is meant by a line by it's first 3 characters is great, will make the parsing even simpler! The ``` case will be a little bit more problematic as the parsing process would need to know what was parsed a few lines before (was a "tag" opened?). Line by line parsing is preferable to allow simple implementations to me, but that's not be a deal-breaker either. From julien at typed-hole.org Sat Jan 18 17:05:35 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 18:05:35 +0100 Subject: Adding a content size meta? Message-ID: Thanks to gemini://konpeito.media I noticed we might be missing the content size information in some cases, mostly binary files. With content that takes some time to be downloaded it would be nice to be able to show the user the download progress. Most clients currently leave the user in the dark regarding what's happening. Since I hope that Gemini will be used for more medium than text as cat uses it, this would be a nice addition to clients. I would imagine this added meta only for non `text/whatever` content. There is also another need I felt being a client developer, it's knowing what mime type an URL will return to be able to treat/display it differently without necessarily getting the content.? Could we imagine a standardized way to query an URL just for it's meta header maybe? These are really client dev concerns and I feel this might be overkill and going against the simplicity we want but I had to ask :) From solderpunk at SDF.ORG Sat Jan 18 17:15:56 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 17:15:56 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118021627.GF3354@brevard.conman.org> References: <20200116222458.GD23953@SDF.ORG> <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <20200118021627.GF3354@brevard.conman.org> Message-ID: <20200118171556.GE4518@SDF.ORG> On Fri, Jan 17, 2020 at 09:16:27PM -0500, Sean Conner wrote: > Red alert! Raise shields! Strap in! This is going to be a bumpy ride. I'd have expected nothing less from you. :) In fact, the very fact that felt this abrupt change of direction was worth careful nit-picking rather than dismisisng it with a fatal flaw up front is a plesant surprise! > Not quite true, even according to this document. Leading white space in > lists is the glaring exception here. I'll deal with this properly in the relevant part below, but there *is* no leading whitespace in lists by definition. "Begins with" means "begins with"! :) > > 5. HEADING LINES > > > > Lines beginning with one, two or three consecutive # characters are > > HEADING LINES, corresponding to headings, subheadings or susubheadings > > respectively. The text following the # characters (with leading > > whitespace removed) > > The parenthetical here is ambiguous. Does it refer to this issue? > > #A title > ## A title with space between the '#' and text > ### Even more white space > > or > > #A title > ##A title with leading space before the '#' > ###Even more white space > > I'm thinking the former now that I'm replying, but my code deals with both > cases combined, so I can handle: Definitely the former. Lines with leading spaces before the # do not satisfy the definition of a HEADING LINE, which is any line that begins with #, ## or ### I'm open to reconsidering the leading whitespace thing. I guess I was thinking that some people might like to line up their headings in this fashion: # Title ## Section ### Sub-section and we should think of the top-level heading as "Title" and not " Title". > Almost---tabs (and yes, I do use tabs---I like me the tabs) are an issue > and while I can handle them (I have code that will expand them up to 8 > spaces) not everybody has code to deal with this. So question: > > WHAT ABOUT TABS? > > They WILL show up. In headings??? I have no idea what to suggest regarding how to handle that. Is this such a deranged edge case that we feel safe letting that behaviour be client-defined? > > constitute the heading and should be displayed to > > the user. Clients MAY choose to apply special styling to headings to > > distinguish them from ordinary lines. However, the primary purpose of > > HEADING LINES is to represent the internal structure of the document > > in a machine-readable way. Advanced clients can use this information > > to, e.g. display a hierarchically formatted "table of contents" for a > > long document in a side-pane, allowing users to easily jump to > > specific sections without excessive scrolling. Or CMS-style tools > > automatically generating menus or Atom/RSS feeds for a directory of > > text/gemini files can use the first heading in a file as a > > human-friendly label for links to it. > > So another question. I have some headings like this: > > ### Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras sodales eget nisi quis condimentum. Donec ipsum arcu, fermentum eu ullamcorper sit amet, facilisis id nunc. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam tempus nulla ut dolor luctus malesuada. Suspendisse orci sem, semper at maximus non, pharetra et justo. Quisque lectus arcu, viverra ac convallis eu, vulputate ut enim. Nulla aliquam, lacus consequat suscipit facilisis, nisl tortor facilisis nisi, vel mattis eros arcu sed tellus. Duis quis lectus pellentesque, posuere dolor ut, sodales massa. Proin vel blandit mauris. > > Given a screen width of 40, which of the four below should be displayed? > > ### Lorem ipsum dolor sit amet, consect > > ### Lorem ipsum dolor sit amet, cons... > > ### Lorem ipsum dolor sit amet, > ### consectetur adipiscing elit. Cras > ### sodales eget nisi quis condimentum. > ### Donec ipsum arcu, fermentum eu > ### ullamcorper sit amet, facilisis id > ### nunc. Class aptent taciti sociosqu > ### ad litora torquent per conubia > ### nostra, per inceptos himenaeos. Nam > ### tempus nulla ut dolor luctus > ### malesuada. Suspendisse orci sem, > ### semper at maximus non, pharetra et > ### justo. Quisque lectus arcu, viverra > ### ac convallis eu, vulputate ut enim. > ### Nulla aliquam, lacus consequat > ### suscipit facilisis, nisl tortor > ### facilisis nisi, vel mattis eros > ### arcu sed tellus. Duis quis lectus > ### pellentesque, posuere dolor ut, > ### sodales massa. Proin vel blandit > ### mauris. > > ### Lorem ipsum dolor sit amet, > consectetur adipiscing elit. Cras > sodales eget nisi quis condimentum. > Donec ipsum arcu, fermentum eu > ullamcorper sit amet, facilisis id > nunc. Class aptent taciti sociosqu > ad litora torquent per conubia > nostra, per inceptos himenaeos. Nam > tempus nulla ut dolor luctus > malesuada. Suspendisse orci sem, > semper at maximus non, pharetra et > justo. Quisque lectus arcu, viverra > ac convallis eu, vulputate ut enim. > Nulla aliquam, lacus consequat > suscipit facilisis, nisl tortor > facilisis nisi, vel mattis eros > arcu sed tellus. Duis quis lectus > pellentesque, posuere dolor ut, > sodales massa. Proin vel blandit > mauris. > > ### Lorem ipsum dolor sit amet, > consectetur adipiscing elit. Cras > sodales eget nisi quis condimentum. > Donec ipsum arcu, fermentum eu > ullamcorper sit amet, facilisis id > nunc. Class aptent taciti sociosqu ad > litora torquent per conubia nostra, per > inceptos himenaeos. Nam tempus nulla ut > dolor luctus malesuada. Suspendisse > orci sem, semper at maximus non, > pharetra et justo. Quisque lectus arcu, > viverra ac convallis eu, vulputate ut > enim. Nulla aliquam, lacus consequat > suscipit facilisis, nisl tortor > facilisis nisi, vel mattis eros arcu > sed tellus. Duis quis lectus > pellentesque, posuere dolor ut, sodales > massa. Proin vel blandit mauris. > > Or is this way into the "you have *got* to be kidding!" territory? I > swear, I'm not trying to take these things to the extreme ... well, okay, I > *am* trying to take these things to the exteme, but only to find out where > the borders are. You're right to do so. Your last example is what would be produced by a simple client which recognises only the compulsory line types (and hence treats heading lines as text lines). I think any of the others would be reasonable outputs for an advanced client, although as a reader I would strongly prefer one of the non-truncating options. I'm not sure it makes sense to specify the correct behaviour here, any more than it makes sense to specify what kind of bullet symbol should be used for list items. > Okay, several questions here. First off, replace the '###' with '*' in > the above example---how to properly format a list item that is ridiculously > long. Well, a simple client that doesn't recognise list items as anything special will just wrap it: * To demonstrate endurance of humans and equipment in spaceflight for extended periods, at least eight days required for a Moon landing, to a maximum of two weeks If we adopt this syntax, I would probably have AV-98 recognise item lines and format the above like this: * To demonstrate endurance of humans and equipment in spaceflight for extended periods, at least eight days required for a Moon landing, to a maximum of two weeks i.e. instead of splitting the line into lines of length N, split them into lines of length N-2, prefix the first line with "* " (or maybe find some nice unicode bullet symbol) and prefix all the other lines with " ". I suspect most clients will do something vaguely similar. I'm not sure we need to constrain this via spec, if somebody writes a client which yields: * To demonstrate endurance of humans and equipment in spaceflight... nobody is going to want to use it. > Second question---this list: > > * lorem > * ipsum > * dolor > * Fusce faucibus ... (I'll save you the text) > * Nunc pharetra ... > * consectetur > * adipiscing > * elit > * Cras > * sodales > * eget > * nisi > > This is a nested list, so the lines start with a blank line. This list > indents one space, but it could be two spaces per index, three, eight. > Also, this list: > > * lorem > * ipsum > * dolor > * Fusce faucibus ... > * Nunc pharetra ... > * consectetur > * adipiscing > * elit > * Cras > * sodales > * eget > * nisi > > Uses tabs (ah, lovely tabs). Are indents preserved? Transformed? What > if there's mixed spaces and tabs (one line is a tab and two space, another > line is 10 spaces---with a default tab stop of 8, they both line up)? Well, as I guess is clear by now, I don't consider most of those lines to be list item lines because they don't start with *. I had intended nested lists to be unsupported, but I don't think I have a problem with Tomasino's idea of permitting them via *** . I'm still thinking on that though. > > 7. ORDERED LIST ITEMS > > > > As above with obvious changes. > > Not so fast there! Again, the same issues as above, but now here are some > ordered lists that are obvious *to me* but maybe not to others: Well, I guess a lot of your questions here are moot in light of my earlier post about the + syntax. > -spc (Will torture specs for food ...) You'll never go hungry! Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 17:23:01 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 17:23:01 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118153947.GD4518@SDF.ORG> Message-ID: <20200118172301.GF4518@SDF.ORG> On Sat, Jan 18, 2020 at 05:41:05PM +0100, Julien Blanchard wrote: > Both my clients try to display content as the author wrote it and I > parse it line by line so I don't see any issue with the proposal as > long as I can get the viewport size in my clients. > Both NCurses and GTK should provide that I'm quite sure. Okay, great! > The ``` case will be a little bit more problematic as the parsing > process would need to know what was parsed a few lines before > (was a "tag" opened?). By "was a tag opened?", do you just mean "am I currently inside or outside a pair of ```s?". I have found this very easy to track, you just need a single boolean variable, initialised to False at the start of the document. Every time you see a ``` line you flip it's value. When processing all other line types, the first thing you do is check whether that value's true. Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 17:30:48 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 17:30:48 +0000 Subject: What shall we call Gemini logs, anyway? Message-ID: <20200118173048.GG4518@SDF.ORG> A post by tfurrows got me thinking about this: gemini://consensus.circumlunar.space/blog/2019-10-24_geminiBlog.gmi (tfurrows, if you're reading - that file is served up as text/plain, which breaks the link at the end!) The web has blogs, gopher has phlogs, what should we call the Gemini equivalent? As tfurrows mentions, a minority of phloggers prefer to use "glog" over "phlog", so that obvious candidate is taken. The next thing that came to my mind is "gemlog", but that feels awkward to me because it's someone's username at SDF (and I follow them on Mastodon, too). But I suppose most people won't recognise it as such. Maybe I'll ask how they'd feel about us using the term. But does anybody have any other ideas? Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 17:34:19 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 17:34:19 +0000 Subject: Adding a content size meta? In-Reply-To: References: Message-ID: <20200118173419.GH4518@SDF.ORG> This was discussed previously in the following thread: https://lists.orbitalfox.eu/archives/gemini/2019/000012.html Which I'm now going to re-read to remind myself of what was said. From memory, I wasn't keen about adding extra fields to the META part of the response header, for my usual fear of opening the gates to people adding on weird non-standard extensions once it's established that more than exactly one thing is allowed there. Interestingly, I somehow forgot about this ubiquitous fear of mine with regards to the recent proliferation of text/gemini line types... Cheers, Solderpunk On Sat, Jan 18, 2020 at 06:05:35PM +0100, Julien Blanchard wrote: > Thanks to gemini://konpeito.media I noticed we might be missing the > content size information in some cases, mostly binary files. > With content that takes some time to be downloaded it would be nice to > be able to show the user the download progress. Most clients currently > leave the user in the dark regarding what's happening. Since I hope > that Gemini will be used for more medium than text as cat uses it, > this would be a nice addition to clients. > I would imagine this added meta only for non `text/whatever` content. > > There is also another need I felt being a client developer, it's > knowing what mime type an URL will return to be able to treat/display > it differently without necessarily getting the content.? Could we > imagine a standardized way to query an URL just for it's meta header > maybe? > > These are really client dev concerns and I feel this might be overkill > and going against the simplicity we want but I had to ask :) From julien at typed-hole.org Sat Jan 18 17:39:47 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 18:39:47 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118172301.GF4518@SDF.ORG> References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118153947.GD4518@SDF.ORG> <20200118172301.GF4518@SDF.ORG> Message-ID: <4730caa8-65b1-a9cd-aa9b-24756288e13e@typed-hole.org> On 1/18/20 6:23 PM, solderpunk wrote: > By "was a tag opened?", do you just mean "am I currently inside or > outside a pair of ```s?". I have found this very easy to track, you > just need a single boolean variable, initialised to False at the start > of the document. Every time you see a ``` line you flip it's value. > When processing all other line types, the first thing you do is check > whether that value's true. Yes, no doubt it's relatively easy to implement but this could open the way to more "tags" and I just hope it won't. I'm really glad this conversation is coming to a conclusion which so far seems to satisfy everybody and we can finally move on to other topics! From solderpunk at SDF.ORG Sat Jan 18 17:48:49 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 17:48:49 +0000 Subject: File size issues In-Reply-To: <20190818113641.GC5354@SDF.ORG> References: <20190815154110.GE28953@SDF.ORG> <20190815180620.GA18238@brevard.conman.org> <20190816104312.GA23118@SDF.ORG> <8736i1z27g.fsf@camilla.carcosa.net> <20190816134427.GA7388@SDF.ORG> <20190818113641.GC5354@SDF.ORG> Message-ID: <20200118174849.GI4518@SDF.ORG> On Sun, Aug 18, 2019 at 11:36:41AM +0000, solderpunk wrote: > Yeah, I think we can leave this for now. It was a hypothetical concern > that somebody had. Not necessarily a bad one, but until it's observed > actually creating significant trouble for actual users on actual clients > I think we can just table this issue. If it does come up as a practical > concern, we can resume discussion of some of the ideas here. Well, I guess this is no longer quite so hypothetical, as Konpeito has given us large binary files over Gemini. I noticed that this was an unpleasant user experience to download via AV-98 (with the client appearing to "hang" for a long time during the download). And, in fact, I did receive an email from one person telling me they liked Gemini a lot, who mentioned that they had not been able to download the Konpeito album. I strongly suspect that, in fact, they just didn't realise their client was slowly downloading it. One possible solution to the user experience problem that doesn't require the client actually knowing the file size is to display, instead of a progress bar, some kind of "stuff is happening" indicator, e.g. an animated spinner (you know, of the /, -, \, | kind) accompanied by a count of KiB / MiB downloaded so far? That would at least make it obvious to users that something was happening. And a convention of specifying filesizes for large binary files in the link text would allow rough mental calculation of progress? Cheers, Solderpunk From julien at typed-hole.org Sat Jan 18 17:50:24 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 18:50:24 +0100 Subject: Adding a content size meta? In-Reply-To: <20200118173419.GH4518@SDF.ORG> References: <20200118173419.GH4518@SDF.ORG> Message-ID: <7d9b3e08-5545-443f-8f01-960e02ace544@typed-hole.org> On 1/18/20 6:34 PM, solderpunk wrote: > This was discussed previously in the following thread: > > https://lists.orbitalfox.eu/archives/gemini/2019/000012.html > > Which I'm now going to re-read to remind myself of what was said. From > memory, I wasn't keen about adding extra fields to the META part of the > response header, for my usual fear of opening the gates to people adding > on weird non-standard extensions once it's established that more than > exactly one thing is allowed there. > > Interestingly, I somehow forgot about this ubiquitous fear of mine with > regards to the recent proliferation of text/gemini line types... Ah yes fogot about that thread! Basically Sean's proposal https://lists.orbitalfox.eu/archives/gemini/2019/000045.html would satisfy all my needs client-wise, but if you prefer not to go this route that's fine too. Kind of a cherry on top of the cake! From b__m__e at mailfence.com Sat Jan 18 17:58:26 2020 From: b__m__e at mailfence.com (Brian Evans) Date: Sat, 18 Jan 2020 18:58:26 +0100 (CET) Subject: Adding a content size meta? Message-ID: <382432062.15242.1579370306771@ichabod.co-bxl> julienxx writes: > Could we imagine a standardized way to query an URL just for it's meta header maybe? I have just been requesting the document reading the first line and then severing the connection. I do agree that having content size, pref in bytes, in the meta would allow for loading animations, percent counters, download managers, etc. Which would benefit the user of any client. --? Sent with https://mailfence.com Secure and private email From solderpunk at SDF.ORG Sat Jan 18 18:34:56 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 18:34:56 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <4730caa8-65b1-a9cd-aa9b-24756288e13e@typed-hole.org> References: <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118153947.GD4518@SDF.ORG> <20200118172301.GF4518@SDF.ORG> <4730caa8-65b1-a9cd-aa9b-24756288e13e@typed-hole.org> Message-ID: <20200118183456.GJ4518@SDF.ORG> On Sat, Jan 18, 2020 at 06:39:47PM +0100, Julien Blanchard wrote: > I'm really glad this conversation is coming to a conclusion which so far > seems to satisfy > everybody and we can finally move on to other topics! Me too! Although I'm still very keen to hear from jmcbray on these recent developments. I *suspect* they'll be happy. But I really don't want to rush into this. But I am amazed at how much more tractable getting away from gopher-style hard wrapping has made this whole discussion. It has made it obvious to me that the problem all along has been with determining the scope of special features. Consider bulleted lists, with hard-wrapping. Suppose we have this content, hard wrapped to 40 characters, and our viewport is 60 characters so we want to combine lines: * To demonstrate endurance of humans and | Line 1 equipment in spaceflight for extended | 2 periods, at least eight days required | 3 for a Moon landing, to a maximum of two | 4 weeks | 5 * To effect rendezvous and docking with | 6 another vehicle, and to maneuver the | 7 combined spacecraft using the | 8 propulsion system of the target vehicle | 9 * To demonstrate Extra-Vehicular | 10 Activity (EVA), or space-"walks" outside | the protection of the spacecraft, and to | evaluate the astronauts' ability to | perform tasks there * To perfect | techniques of atmospheric reentry and | touchdown at a pre-selected location on | land | Any old client can recognise that Line 1 is a list item by virtue of the leading *. But in order for a client to do a nice job of reformatting this list to 60 chars, it has to understand that lines 1 through 5 constitute a single list item (i.e. that this is the scope of the * in line 1), and that lines 2-5 are not an independent paragraph of text coming after a list with only a single item. To a human, this is obvious, even if they can't read English, because the list items at lines 6 and 10 make it clear. But to a parser, at the time it encounters line 2, it has no idea what's in lines 6 or 10. So parsing requires looking ahead, and understanding rules like "Once started, a list item continues until we encounter either a new list item or a blank line". It's exactly that kind of fiddly rule that I was terrified of filling the spec with earlier. In HTML, the scoping problem is trivial, because there are explicit beginning and tags,
  • and
  • . Markdown gets rid of these for visual lightness (great!), but that scoping information is still essential. In Markdown it's in there implicitly for a sufficiently complicated (bad!) parser which understands the concept of blocks (which can be nested). So Markdown trades the visual complexity of HTML for conceptual complexity that makes it a pain to parse. The syntax we're now talking about, where scope is limited to a single line, is such an elegant alternative to both of these! One way to think about it is that things like =>, * and # are differnet kinds of opening tag but they all have the same closing tag, which is \n. Because there are opening and closing tags, the parsing is easy, like HTML. But because our closing tags are invisible and need to be there anyway, we get ease of authoring and visual lightness like Markdown. It's the best of both worlds. I'm so glad we hit upon this before I ran out of patience and just declared Gopher-style plain text hard wrapping to be the final decision! Thanks to everybody who steered the conversation in this direction. Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 18:45:30 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 18:45:30 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> Message-ID: <20200118184530.GK4518@SDF.ORG> On Sat, Jan 18, 2020 at 09:46:47AM -0500, Dave Gauer wrote: > > Hello! I've been lurking throughout the entire long history of this > discussion. ("Wanted to contribute, but other commitments...etc. etc.") > > I just wanted to pipe in to share my excitement about this simple 3-char > rule. I *love* the simplicity of "context-free" line-based parsing. I'm so glad to hear this from you in particular! Reading your phlog convinced me that you've spent more time thinking about markup and formatting issues for plain text than most other people talking about it combined. If you think we're on a good track here, I'm feeling a lot better. Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 18:57:49 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 18:57:49 +0000 Subject: Adding a content size meta? In-Reply-To: <382432062.15242.1579370306771@ichabod.co-bxl> References: <382432062.15242.1579370306771@ichabod.co-bxl> Message-ID: <20200118185749.GL4518@SDF.ORG> On Sat, Jan 18, 2020 at 06:58:26PM +0100, Brian Evans wrote: > julienxx writes: > > Could we imagine a standardized way to query > an URL just for it's meta header maybe? > > I have just been requesting the document reading > the first line and then severing the connection. > > > I do agree that having content size, pref in bytes, in > the meta would allow for loading animations, > percent counters, download managers, etc. Which > would benefit the user of any client. I guess the question is whether we prefer adding this information into the response header and having clients do the request-and-terminate trick, or defining a well-known endpoint servers can implement which takes a path as an input and returns either what the response header would be, or maybe a small JSON structure of metadata (which could include stuff like last-modified date). Cheers, Solderpunk From solderpunk at SDF.ORG Sat Jan 18 19:05:19 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 19:05:19 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <660019441.8880.1579116650232@ichabod.co-bxl> References: <660019441.8880.1579116650232@ichabod.co-bxl> Message-ID: <20200118190519.GM4518@SDF.ORG> On Wed, Jan 15, 2020 at 08:30:50PM +0100, Brian Evans wrote: > Solderpunk wrote recently regarding use of color in gemini > documents. > > I would like to put in a vote to the contrary. Nearly forgot to get around to replying to this, but not quite. :) When I previously declared I wasn't interested in supporting colour, I was thinking purely about defining some kind of syntax for it, like this (or any similar such thing). I didn't mean to say anything about ANSI escape codes. I know we're going to have to come up with some kind of official stance on those eventually, but I've given it no serious thought yet. Cheers, Solderpunk From julien at typed-hole.org Sat Jan 18 19:20:31 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 20:20:31 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118190519.GM4518@SDF.ORG> References: <660019441.8880.1579116650232@ichabod.co-bxl> <20200118190519.GM4518@SDF.ORG> Message-ID: On 1/18/20 8:05 PM, solderpunk wrote: > I didn't mean to say anything about ANSI escape codes. I know we're > going to have to come up with some kind of official stance on those > eventually, but I've given it no serious thought yet. FWIW I intend to add some stuff with ANSI colors on my Gemini-space, I have some ideas I need to tryout :) I might add two links though, one for the colored version and another for the readable one if needed. Since Bombadillo and Castor already support ANSI colors and kompeito.media is such a great place, I'm in favor of officially supporting them or at least not ban them, up to the clients to deal with them or not. From aaron at ajanse.me Sat Jan 18 19:20:10 2020 From: aaron at ajanse.me (Aaron Janse) Date: Sat, 18 Jan 2020 11:20:10 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118184530.GK4518@SDF.ORG> References: <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118184530.GK4518@SDF.ORG> Message-ID: I think we made an oversight: syntax nested within quotes. For example: ```gemini > > It looks like things are moving along, I wonder if we forgot anything > > that would make us need to read more than three chars to specify the > > line type? > > Here's a site to demo gemini's syntax: > => gemini://example.com Gemini Syntax Demo > > In the future, we can add the following features: > * foo > * bar Client: *doesn't know to make quoted link clickable* *doesn't know to fancy-render the quoted list* ``` --- Regardless, here are my thoughts on everything else... I definitely love: * The use of asterisks only for unordered lists * The use of plus signs for ordered lists. I wish markdown had this! * Repeated list markers (*/+) to indicate depth * Stateless line-by-line parsing Changes I think we should make to the spec spec, based on what I've read here and my own opinion: * Preformatted text toggle lines only need to *start* with three backticks. Specify the significance of text after the backticks. Some ideas: * It's permitted in order to please markdown users. Ignore it * Display it to the user (could be position in file, language, etc) * Use it for syntax highlighting * Specify that preformatted code blocks are intended for content such as ascii art and code, meaning that it should be easily copy-pasteable into a text editor without needed to undergo extra steps to revert it from its displayed form to its original form * Add horizontal rule lines (three+ dashes) * Specify that ordered lists MUST use plus sign markers * Specify Tomasino's nested list system * Explicitly specify markdown syntax that is not allowed. Random idea, but maybe we could even advise clients to shame this syntax the same way modern web browsers are shaming non-HTTP sites? Regardless, here are some things that I think we should explicitly ban in text/gemini: * Ordered lists using letters or numberal markers * Indented markdown lists * Unordered lists starting with a dash * Embedded HTML * Markdown images * Tables in non-preformatted text blocks * Hard-wrapping quotes * Hard-wrapping text * Horizontal rules with anything other than three+ dashes * Expecting anything special when a line ends with whitespace Are we really limited to a max depth of three? Even if we allow unlimited depth of headers and lists, clients would only need to read the first two chars of a line to determine its type (unless we add horizontal rules, in which case we'd need to read three characters). > Hmm. I would want to think a bit before I lay down a hard statement on > this because I don't want to impose too much of my own ideology on > Gemini, as it's supposed to be a general-purpose tool....but I am not > excited by verbatim dragging of mainstream web 2.0 cultural concepts > like comment threads into Gemini. Well, worst case scenario, if someone really badly wants comment threads, maybe they could use nested quote blocks (assuming we figure that out). Cheers! From aaron at ajanse.me Sat Jan 18 19:24:12 2020 From: aaron at ajanse.me (Aaron Janse) Date: Sat, 18 Jan 2020 11:24:12 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <660019441.8880.1579116650232@ichabod.co-bxl> <20200118190519.GM4518@SDF.ORG> Message-ID: > Since Bombadillo and Castor already support ANSI colors and > kompeito.media is such a great place, I'm in favor of officially > supporting them or at least not ban them, up to the clients to deal with > them or not. Hmmm. It does seem, though, that *allowing* ANSI colors would require non-terminal clients to strip ANSI colors, which would be a PITA, expecially considering that ANSI is a hot mess (I built an ANSI parser a while ago [1]) => https://github.com/aaronjanse/i3-tmux/tree/master/vterm [1] From julien at typed-hole.org Sat Jan 18 20:08:27 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sat, 18 Jan 2020 21:08:27 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: Message-ID: <0F5F2C9A-711B-4C23-9C3D-5336CE46B458@typed-hole.org> I thought the use case of ``` was precisely do nothing with the next lines (ie. don?t parse) until marker is met again or am I wrong here? From aaron at ajanse.me Sat Jan 18 20:30:32 2020 From: aaron at ajanse.me (Aaron Janse) Date: Sat, 18 Jan 2020 12:30:32 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <0F5F2C9A-711B-4C23-9C3D-5336CE46B458@typed-hole.org> References: <0F5F2C9A-711B-4C23-9C3D-5336CE46B458@typed-hole.org> Message-ID: <31870ce6-c0dd-4fd9-a8cb-c8324fa2658c@www.fastmail.com> I meant to say "text after the ticks, but in the same line." Such as: ```this text Code code code ```or this text From solderpunk at SDF.ORG Sat Jan 18 21:33:03 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 21:33:03 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118184530.GK4518@SDF.ORG> Message-ID: <20200118213303.GN4518@SDF.ORG> On Sat, Jan 18, 2020 at 11:20:10AM -0800, Aaron Janse wrote: > I think we made an oversight: syntax nested within quotes. > > Client: > *doesn't know to make quoted link clickable* > *doesn't know to fancy-render the quoted list* > ``` Hmm, I really would not have had any expectation that quoted Gemini syntax would do anything at all. What sort of context would you expect this to occur in? > * Preformatted text toggle lines only need to *start* with three > backticks. Specify the significance of text after the backticks. I think I'm probably fine with this, does anybody have objections? > * Specify that preformatted code blocks are intended for content such as > ascii art and code, meaning that it should be easily copy-pasteable > into a text editor without needed to undergo extra steps to revert it > from its displayed form to its original form Yeah, it might be a good idea to emphasise the intent to preserve copy-and-pastability. > * Add horizontal rule lines (three+ dashes) I guess this is harmless. It feels a bit to me like we're adding it just because Markdown has it - unlike headings and lists and even, occasionally, quotes, I don't know that I've ever seen a horiozontal rule used in Gopherspace. But I don't see a good reason to disallow it. > * Specify that ordered lists MUST use plus sign markers Yep. > * Specify Tomasino's nested list system I think I'm still onboard with this, although I'm starting to wonder about how these nested lists will look when rendered by a basic client treating them as text lines. I'm not sure it degrades to something terribly readable. > * Explicitly specify markdown syntax that is not allowed. It feels very strange to me for a syntax specification to explicitly list stuff from a different syntax specification which isn't allowed. I can see it being helpful to point this stuff out in a tutorial for people learning text/gemini, but in a formal specification of a markup format, it goes without saying that anything which isn't explicitly supported is unsupported. > but maybe we could even advise clients to shame this syntax the same > way modern web browsers are shaming non-HTTP sites? Wouldn't doing that (all questions about whether this is appropriate behaviour aside) require writing code to detect all the stuff that we're not supporting precisely because it's a pain to write code to reliably detect it? Seems counterproductive! > Regardless, here > are some things that I think we should explicitly ban in text/gemini: > ... > * Hard-wrapping text I don't want to explicitly ban hard-wrapped text, I don't see the need to. I think this syntax actually degrades pretty gracefully when fed hard-wrapped text that is shorter than the viewport, and that's nice. I think the vast majoity of people will end up taking the long line approach because it will support a wider range of clients (especially narrow screens) and some things will render slightly nicer. If a small percentage want to stick to the old ways for whatever reason, knowing and accepting the downsides, I see no reason not to let them. > Are we really limited to a max depth of three? Even if we allow unlimited > depth of headers and lists, clients would only need to read the first two > chars of a line to determine its type (unless we add horizontal rules, > in which case we'd need to read three characters). Good catch, technically speaking once a line is detected, on the basis of the first three or fewer chars, as a header or list, it can be passed to a function than handles a header line or a list line, and that function has access to the whole line. That said, maybe we should add a limit anyway. Otherwise clients have to write totally generalised code to handle arbitrarily many levels, which could get tricky. > Well, worst case scenario, if someone really badly wants comment threads, > maybe they could use nested quote blocks (assuming we figure that out). Well, it seems like the > syntax generalises in exactly the same way as the heading and list syntaxes. Speaking of these...what happens when a client encounters this: ** Foo ** Bar ** Baz i.e. a bunch of allegedly nested list items which are not emedded in a higher-level list? For a simple text-based client which only uses the nestedness level to determine how many spaces to stick in front of the line, this shouldn't post major problems. But fancy clients planning on doing graphical things (I'm actually thinking more about quotes here than list items) might choke if not carefully written. Should we explicitly require correct nesting? Or explicitly say clients must be resilient against weird nesting? Cheers, Solderpunk From b__m__e at mailfence.com Sat Jan 18 21:34:27 2020 From: b__m__e at mailfence.com (Brian Evans) Date: Sat, 18 Jan 2020 22:34:27 +0100 (CET) Subject: Text reflow woes (or: I want bullets back!)y Message-ID: <739591427.21203.1579383267196@ichabod.co-bxl> Aaron Janse writes: > Hmmm. It does seem, though, that *allowing* ANSI colors would require > non-terminal clients to strip ANSI colors, which would be a PITA, > expecially considering that ANSI is a hot mess (I built an ANSI parser > a while ago [1]) Currently Bombadillo has a few different modes. The normal mode removes ansi escape codes. As I am parsing a document if I read an `\033` character I just toggle an escape code boolean and then consume until I read a A-Za-z character (and consume that char as well). It works very quickly and handles removing them quite well. I do the same thing for the color mode for any escape codes that do not end in `m`. That said, it may not work as well for people not parsing by writing characters into a buffer char by char. I would also argue that it would _not_ require clients to strip them. There are a few options: 1. Decide, rightly in my opinion, that if a content creator uses escape codes they are taking the chance that the codes themselves will be displayed to the eventual viewer depending on the client. 2. Do a simple find and replace on the whole document for '\033' and replace it with "ESC". While this will still leave the codes displaying to the viewer they will not actually render, thus you do not need to worry about line movement, screen clears, etc. --? Sent with https://mailfence.com Secure and private email From aaron at ajanse.me Sat Jan 18 22:18:39 2020 From: aaron at ajanse.me (Aaron Janse) Date: Sat, 18 Jan 2020 14:18:39 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118213303.GN4518@SDF.ORG> References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118184530.GK4518@SDF.ORG> <20200118213303.GN4518@SDF.ORG> Message-ID: On Sat, Jan 18, 2020, at 1:33 PM, solderpunk wrote: > Hmm, I really would not have had any expectation that quoted Gemini > syntax would do anything at all. > > What sort of context would you expect this to occur in? I often see quoted syntax (e.g. links, lists) on StackOverflow. However, I haven't seen quoted syntax in phlogs. Plus, supported mixing nested syntax could be a PITA to implement. Also seen in the wild: * nested lists of links (nice to have, but not sad if unsupported) * hyperlink headers (this sounds like a mess regarding `=> #` vs `# =>`) Either way, I'd still specify what nesting is/isn't allowed. > I guess this is harmless. It feels a bit to me like we're adding it > just because Markdown has it - unlike headings and lists and even, > occasionally, quotes, I don't know that I've ever seen a horiozontal > rule used in Gopherspace. But I don't see a good reason to disallow it. I see horizontal rules used in gopherspace all the time: => gopher://zaibatsu.circumlunar.space/0/~solderpunk/phlog/a-new-start.txt => gopher://zaibatsu.circumlunar.space/0/~tfurrows/phlog/2020-01-02_clickBaitGeneration.txt => gopher://tilde.team/0/~ben/blog/weechat-setup.md > It feels very strange to me for a syntax specification to explicitly > list stuff from a different syntax specification which isn't allowed. You make a good point here. Maybe, we instead specify that clients MUST NOT support non-specified syntax, in order to discourage pages with un-specified syntax from being published in the first place. > I don't want to explicitly ban hard-wrapped text, I don't see the need > to. I think this syntax actually degrades pretty gracefully when fed > hard-wrapped text that is shorter than the viewport, and that's nice. That makes sense. However, would it be unreasonable to state that users "SHOULD NOT" hard-wrap, since doing so degrades quite poorly on mobile? > > > Are we really limited to a max depth of three? Even if we allow unlimited > > depth of headers and lists, clients would only need to read the first two > > chars of a line to determine its type (unless we add horizontal rules, > > in which case we'd need to read three characters). > > Good catch, technically speaking once a line is detected, on the basis > of the first three or fewer chars, as a header or list, it can be passed > to a function than handles a header line or a list line, and that > function has access to the whole line. > > That said, maybe we should add a limit anyway. Otherwise clients have > to write totally generalised code to handle arbitrarily many levels, > which could get tricky. Hmmm. If max depth is one, that makes sense. Same if max depth is two. But once max depth is three, it sounds like clients would be using generalized code anyway. Plus, a shallow max depth might be very confusing for authors using text/gemini. > > > Well, worst case scenario, if someone really badly wants comment threads, > > maybe they could use nested quote blocks (assuming we figure that out). > > Well, it seems like the > syntax generalises in exactly the same way as > the heading and list syntaxes. But for quotes, I beg that a space character is allowed between the greater-than symbols in nested quotes, so that we don't have to abandon the way quotes are used in emails. In fact, I think the spec should say that a single space after a `>` symbol should always be stripped. > > Speaking of these...what happens when a client encounters this: > > ** Foo > ** Bar > ** Baz > > i.e. a bunch of allegedly nested list items which are not emedded in a > higher-level list? > > For a simple text-based client which only uses the nestedness level to > determine how many spaces to stick in front of the line, this shouldn't > post major problems. But fancy clients planning on doing graphical > things (I'm actually thinking more about quotes here than list items) > might choke if not carefully written. Should we explicitly require > correct nesting? Or explicitly say clients must be resilient against > weird nesting? I personally think resilients is a bad idea. It's partially to blame for so many web browser quirks nowadays. I think that if fancy clients refuse to render invalid syntax, that could be a good deterrant. OTOH, it's sometimes tough to contact gopher/gemini users, so maybe nobody would be able to tell them something's wrong. Unless we write scripts to verify that text is valid text/gemini, and/or integrate this functionality into servers? Cheers! From sean at conman.org Sat Jan 18 22:35:08 2020 From: sean at conman.org (Sean Conner) Date: Sat, 18 Jan 2020 17:35:08 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118153947.GD4518@SDF.ORG> Message-ID: <20200118223508.GH3354@brevard.conman.org> It was thus said that the Great Julien Blanchard once stated: > On 1/18/20 4:39 PM, solderpunk wrote: > > >Julien, as the author of multiple Gemini clients, how do you feel about > >the prospect of most text/gemini content having very long lines which > >need to be wrapped to fit the viewport? > > The ``` case will be a little bit more problematic as the parsing > process would need to know what was parsed a few lines before > (was a "tag" opened?). I found that to be easy to support. The code looks like: local literal = false for line in file:lines() do if line:match "^#" then ... elseif line:match "^```$" then literal = not literal -- Yup, that's all there was to it. elseif line:match "^=>" then ... else if literal then display_text_to_width(line) else wrap_text_to_width(line) end end -spc From sean at conman.org Sat Jan 18 23:10:09 2020 From: sean at conman.org (Sean Conner) Date: Sat, 18 Jan 2020 18:10:09 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118171556.GE4518@SDF.ORG> References: <142b420d-acf6-48c0-8aca-21c4348a7ad3@www.fastmail.com> <20200117111158.GA10236@SDF.ORG> <20200117133312.GA7402@SDF.ORG> <20200117135804.GA5259@briz> <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <20200118021627.GF3354@brevard.conman.org> <20200118171556.GE4518@SDF.ORG> Message-ID: <20200118231009.GI3354@brevard.conman.org> It was thus said that the Great solderpunk once stated: > On Fri, Jan 17, 2020 at 09:16:27PM -0500, Sean Conner wrote: > > > Red alert! Raise shields! Strap in! This is going to be a bumpy ride. > > I'd have expected nothing less from you. :) In fact, the very fact that > felt this abrupt change of direction was worth careful nit-picking > rather than dismisisng it with a fatal flaw up front is a plesant > surprise! Well, I don't have very strong feelings about text formatting for Gemini. Also, anything that's been cleared up in subsequent emails will not be addressed here as there's no need to clutter things up. > > Almost---tabs (and yes, I do use tabs---I like me the tabs) are an issue > > and while I can handle them (I have code that will expand them up to 8 > > spaces) not everybody has code to deal with this. So question: > > > > WHAT ABOUT TABS? > > > > They WILL show up. > > In headings??? I have no idea what to suggest regarding how to handle > that. Is this such a deranged edge case that we feel safe letting that > behaviour be client-defined? Not necessarily in headers, but they will appear in Gemini files (I'm thinking most likely in pre-formatted blocks). The thing the clients have to be aware of is that tabs take 1 character space, but can reference up to N spaces (where N is typically 8, but can be arbitrary values per tab). > Well, I guess a lot of your questions here are moot in light of my > earlier post about the + syntax. They are, and I shall update my code accordingly. > > -spc (Will torture specs for food ...) > > You'll never go hungry! Ha! -spc From tomasino at lavabit.com Sat Jan 18 23:13:50 2020 From: tomasino at lavabit.com (James Tomasino) Date: Sat, 18 Jan 2020 23:13:50 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117142715.GA23881@SDF.ORG> <20200117160708.GA11117@briz> <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118184530.GK4518@SDF.ORG> <20200118213303.GN4518@SDF.ORG> Message-ID: On 1/18/20 10:18 PM, Aaron Janse wrote: > Speaking of these...what happens when a client encounters this: > > ** Foo > ** Bar > ** Baz > > i.e. a bunch of allegedly nested list items which are not emedded in a > higher-level list? Line 1: A list has started at depth 2. Display a depth 2 list item. Line 2: A list has continued at depth 2. Display another depth 2 list item. Line 3: A list has continued at depth 2. Display another depth 2 list item. Pretty straight forward if we're just processing line by line. It's every-so-slightly trickier if we're working in ordered lists. You'll want to keep an ordinal stack. + item 1 ++ sub item 1 +++ sub-sub item 1 +++ sub-sub item 2 ++ sub item 2 +++ a new sub-sub item 1 ++ sub item 3 + item 2 As you push deeper you'll want to keep references to those items at a higher level so you can continue to number when you pop back out. Upon popping, though, you can reset the deeper list ordinal. At least, that's how I'd handle it. What constitutes the end of an ordered list? Unordered lists are "dumb" in that they just display their appropriate depth no matter what, but the ordered lists need to keep track of that state. Does a non-list line break the list? What about empty lines? + is this list + still counting + up to 3? From solderpunk at SDF.ORG Sat Jan 18 23:22:39 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sat, 18 Jan 2020 23:22:39 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <20200117172902.GA1432@SDF.ORG> <8517d9cd-e722-4301-aa7a-304412bdb18f@www.fastmail.com> <20200118114503.GA4518@SDF.ORG> <0913EE1C-BCF8-40AE-BF2B-188DCA09310D@lavabit.com> <20200118184530.GK4518@SDF.ORG> <20200118213303.GN4518@SDF.ORG> Message-ID: <20200118232239.GB29273@SDF.ORG> On Sat, Jan 18, 2020 at 11:13:50PM +0000, James Tomasino wrote: > What constitutes the end of an ordered list? Unordered lists are "dumb" > in that they just display their appropriate depth no matter what, but > the ordered lists need to keep track of that state. Does a non-list line > break the list? What about empty lines? > > + is this list > > + still counting > > + up to 3? Argh, excellent question. It seems like something vaguely like the notion of a block has crept in! This is a timely question, too. I'm goofing around trying to implement some of these new ideas in AV-98 (and so far I'm very pleased with the results - # headers are made bold with ANSI escape codes, and unordered lists get nice bullets and nice spacing, looks great!) and had to deal with exactly this. For testing I just decided that anything other than an ordered list line breaks a list and resets the counter. I'm open to other ideas but I worry that anything other than this is liable to be too complicated. Cheers, Solderpunk From julien at typed-hole.org Sat Jan 18 23:36:19 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sun, 19 Jan 2020 00:36:19 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200118232239.GB29273@SDF.ORG> References: <20200118232239.GB29273@SDF.ORG> Message-ID: <1226FC48-8721-4396-B306-8601007FE468@typed-hole.org> > Le 19 janv. 2020 ? 00:22, solderpunk a ?crit : > > ?Argh, excellent question. It seems like something vaguely like the > notion of a block has crept in! > > This is a timely question, too. I'm goofing around trying to implement > some of these new ideas in AV-98 (and so far I'm very pleased with the > results - # headers are made bold with ANSI escape codes, and unordered > lists get nice bullets and nice spacing, looks great!) and had to deal > with exactly this. For testing I just decided that anything other than > an ordered list line breaks a list and resets the counter. I'm open to > other ideas but I worry that anything other than this is liable to be > too complicated. > Do we really need ordered lists? I?m not sure the what the use case is. Couldn?t they be replaced by: ### 1. Foo ### 2. Bar Or something like that if you really wanted numbers? From sean at conman.org Sat Jan 18 23:36:37 2020 From: sean at conman.org (Sean Conner) Date: Sat, 18 Jan 2020 18:36:37 -0500 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <739591427.21203.1579383267196@ichabod.co-bxl> References: <739591427.21203.1579383267196@ichabod.co-bxl> Message-ID: <20200118233637.GJ3354@brevard.conman.org> It was thus said that the Great Brian Evans once stated: > Aaron Janse writes: > > Hmmm. It does seem, though, that *allowing* ANSI colors would require > > non-terminal clients to strip ANSI colors, which would be a PITA, > > expecially considering that ANSI is a hot mess (I built an ANSI parser > > a while ago [1]) > > Currently Bombadillo has a few different modes. The normal mode removes > ansi escape codes. As I am parsing a document if I read an `\033` character I > just toggle an escape code boolean and then consume until I read a A-Za-z > character (and consume that char as well). It works very quickly and handles > removing them quite well. I do the same thing for the color mode for any > escape codes that do not end in `m`. That said, it may not work as well for > people not parsing by writing characters into a buffer char by char. Having written an ECMA-48 (the terminal control codes everybody calls ANSI escape codes when they aren't defined by ANSI) parser you'll probably catch 99% of the control codes used. But the actual definition is (RFC-5234 BNF): CSI = %d27 '[' / %d155 ; ISO-8859-1 or similar / %d194 %d155 ; UTF-8 encoding param = %d48-63 ; chars '0' through '?' meta = %d32-47 ; chars ' ' through '/' cmd = %d64-126 ; chars '@' through '~' sequence = CSI *param *meta cmd There are other ECMA-48 sequences that could prove dangerous if not filtered for. I do have Lua code to parse these [1][2] and use them in my current gopher client to filter them out (and yes, I have come across sites that embed ECMA-48 control codes). > 2. Do a simple find and replace on the whole document for '\033' and replace > it with "ESC". While this will still leave the codes displaying to the viewer > they will not actually render, thus you do not need to worry about line > movement, screen clears, etc. You might want to replace the following codepoints to render control codes harmless: 0 - 31 ; C0 set, except interpret the range from 7-13 inclusive 127 ; DEL 128-159 ; C1 set I say codepoints because in UTF-8, the C1 set is represented by the sequences 194 128 through 194 129 -spc [1] https://github.com/spc476/LPeg-Parsers/blob/master/iso/control.lua This handles encodings in ISO-8859-1 and similar. I have a UTF-8 one that is separate. This one just returns the escape sequence as a unit with no further parsing of the actual sequence. [2] https://github.com/spc476/LPeg-Parsers/blob/master/iso/ctrl.lua This does a more complete parse of the escape sequence, to include its name (if any). Again, This is for ISO-8859-1 and similar encodinds. I have another version for UTF-8. From tiwesdaeg at tilde.pink Sat Jan 18 23:40:12 2020 From: tiwesdaeg at tilde.pink (tiwesdaeg at tilde.pink) Date: Sat, 18 Jan 2020 23:40:12 +0000 Subject: What shall we call Gemini logs, anyway? In-Reply-To: <20200118173048.GG4518@SDF.ORG> References: <20200118173048.GG4518@SDF.ORG> Message-ID: <20200118234012.r4f3ro2eempk6hez@tilde.pink> On Sat, Jan 18, 2020 at 05:30:48PM +0000, solderpunk wrote: > A post by tfurrows got me thinking about this: > > gemini://consensus.circumlunar.space/blog/2019-10-24_geminiBlog.gmi > > (tfurrows, if you're reading - that file is served up as text/plain, > which breaks the link at the end!) > > The web has blogs, gopher has phlogs, what should we call the Gemini > equivalent? > > As tfurrows mentions, a minority of phloggers prefer to use "glog" over > "phlog", so that obvious candidate is taken. > > The next thing that came to my mind is "gemlog", but that feels awkward > to me because it's someone's username at SDF (and I follow them on > Mastodon, too). But I suppose most people won't recognise it as such. > Maybe I'll ask how they'd feel about us using the term. > > But does anybody have any other ideas? > > Cheers, > Solderpunk I've been using the term gemlog, but I like the word gem and don't think gemlog sounds awkward. I think glog sounds weird. It makes me think of the word gl?gg. Tiwesdaeg From julien at typed-hole.org Sun Jan 19 00:04:44 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sun, 19 Jan 2020 01:04:44 +0100 Subject: What shall we call Gemini logs, anyway? In-Reply-To: <20200118173048.GG4518@SDF.ORG> References: <20200118173048.GG4518@SDF.ORG> Message-ID: <367CB83F-81E0-4E35-BAAF-B3C8D8EF63C8@typed-hole.org> I used ?Flight Journal? for mine because gemlog reminds me too much of Ruby gems, but that?s not really something I would advise. glog sounds a bit weird too, at least when trying to pronounce it. Looking at the Gemini program I found the word ?capsule?. What?s interesting is that in French it can mean ?a short column on a given subject? and I kinda like how it sounds. From sean at conman.org Sun Jan 19 00:10:19 2020 From: sean at conman.org (Sean Conner) Date: Sat, 18 Jan 2020 19:10:19 -0500 Subject: Color and other escape sequences in Gemini Message-ID: <20200119001019.GK3354@brevard.conman.org> I'm seeing some people wanting to support some other formatting options like color, leading to the support of ECMA-48 [1]. I would rather NOT see this supported because of several issues. 1). Complexity. The spec is rather complex and wide ranging. There are 130 escape codes (and I'm EXCLUDING those defined by ANSI, codes 0 through 31). If you are filtering them, then five or six cases (depending if you want to support one non-standard encoding used by xterm) need to be considered (which is still four or five more than most would expect). 2). Security. Passing raw escape sequences can not only leave the termainal into an unknown state, but there are a few seqences that are especially alarming: DCS Device Control String OSC Operating System Command APC Application Program Command which, if supported, are exactly what they say on the tin, and thus, you *don't* want these to be processed *at all*. Personally, I haven't come across any terminal or program that supports these. Also, ANSI.SYS, originally for MS-DOS but maybe it still exists for Windows? (I don't know, I don't use Windows) allows one to redefine any key on the keyboard to send a new sequence. "DELTREE C:\*" anyone? Below is the BNF (RFC-5234) for ECMA-48 control sequences: CSI = %d27 '[' / %d155 ; ISO charset / %d194 %d155 ; UTF-8 OSC = %d27 ']' / %d157 ; ISO / %d194 %d157 ; UTF-8 ST = %d27 '\' / %d156 ; ISO / %d194 %156 ; UTF-8 string = %d27 ( 'P' / 'X' / '^' / '_') / (%d144 / %d152 / %d158 / %d159) ; ISO / %d194 (%d144 / %d152 / %d158 / %d159) ; UTF-8 iso = %d160-255 ; the utf8 rule is any UTF-8 codepoint 160 or higher sequence = CSI *(%d48-63) *(%d32-47) %d64-126 / OSC *(%d8-13 / %32-126 / iso / utf8) (ST / %d7) / string *(%d8-13 / %32-126 / iso / utf8) ST / %d27 (%d64-126) / %d128-159 ; ISO / %d194 (%d128-159) ; UTF-8 If you are parsing an ISO charset (like ISO-8859-1) then remove the rules marked UTF-8; similarly, if you are parsing UTF-8, then remove the rules marked ISO. -spc [1] The so called 'ANSI escape codes', which is funny because they're not defined by ANSI, but ISO, so of course they're called ECMA. From aaron at ajanse.me Sun Jan 19 00:14:27 2020 From: aaron at ajanse.me (Aaron Janse) Date: Sat, 18 Jan 2020 16:14:27 -0800 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <1226FC48-8721-4396-B306-8601007FE468@typed-hole.org> References: <20200118232239.GB29273@SDF.ORG> <1226FC48-8721-4396-B306-8601007FE468@typed-hole.org> Message-ID: <051fb7d9-f8c4-4d87-93b6-b6f70498d6ed@www.fastmail.com> On Sat, Jan 18, 2020, at 3:36 PM, Julien Blanchard wrote: > > > Le 19 janv. 2020 ? 00:22, solderpunk a ?crit : > > > > ?Argh, excellent question. It seems like something vaguely like the > > notion of a block has crept in! > > > > This is a timely question, too. I'm goofing around trying to implement > > some of these new ideas in AV-98 (and so far I'm very pleased with the > > results - # headers are made bold with ANSI escape codes, and unordered > > lists get nice bullets and nice spacing, looks great!) and had to deal > > with exactly this. For testing I just decided that anything other than > > an ordered list line breaks a list and resets the counter. I'm open to > > other ideas but I worry that anything other than this is liable to be > > too complicated. > > > > Do we really need ordered lists? I?m not sure the what the use case is. Some use cases: 1. Instructions with step numbers. Yes, references to steps may need to be changed if items are items, but that's less work than replacing both references and the bullet markers themselves. 2. Listing things (e.g. problems), then referring to them by number ("but this once again causes problem #1") 3. "There are X ways to do this... " I don't know what to do about spaces between ordered lists. If there's text between items, I think manually-numbered sections should be used instead. But if it's only newlines between items, I don't see why the count should be restarted. Cheers! From julien at typed-hole.org Sun Jan 19 00:24:59 2020 From: julien at typed-hole.org (Julien Blanchard) Date: Sun, 19 Jan 2020 01:24:59 +0100 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: <051fb7d9-f8c4-4d87-93b6-b6f70498d6ed@www.fastmail.com> References: <051fb7d9-f8c4-4d87-93b6-b6f70498d6ed@www.fastmail.com> Message-ID: Sorry I wasn?t clear, I meant what would be the use case of showing them in a special way compared to headers with a number or plain text? From tomasino at lavabit.com Sun Jan 19 01:47:44 2020 From: tomasino at lavabit.com (James Tomasino) Date: Sun, 19 Jan 2020 01:47:44 +0000 Subject: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <051fb7d9-f8c4-4d87-93b6-b6f70498d6ed@www.fastmail.com> Message-ID: <6D0FC1A7-2676-4AE9-BB1B-D21E4FDA377A@lavabit.com> -------- Original Message -------- From: Julien Blanchard Sorry I wasn?t clear, I meant what would be the use case of showing them in a special way compared to headers with a number or plain text? ---- 1. Listing things 2. Quick instructions where headers are overkill 3. Track listings 4. Top 10 lists 5. The same reasons we want bullets the reflow 6. Recipes! And so on. From b__m__e at mailfence.com Sun Jan 19 19:15:14 2020 From: b__m__e at mailfence.com (Brian Evans) Date: Sun, 19 Jan 2020 20:15:14 +0100 (CET) Subject: FW: Text reflow woes (or: I want bullets back!)y Message-ID: <574762512.1104.1579461314537@ichabod.co-bxl> I've mostly been sitting back through all of this list and raw mode talk, only occasionally popping in small points here or there. But I figure if this is a community decision it is good if those in the community voice their support or lack of support for ideas. I am mostly ambivalent about the need for a raw text mode. Terminals and any graphical clients (GTK, FLTK, Tcl/Tk) all wrap to the window at either a hard or a word wrap. If we arent reflowing, then we are already doing what needs to be done. There has been talk about not wrapping python code. Ok. I get how that would look bad. But which is more readable: 1. Hard-wrapped python code 2. Truncated python code where you just straight up lose the content Clearly #1. I get it as a nice thing for ascii art, but again I feel like using ascii art is a risk a content creator takes... knowing that it may degrade poorly on various output systems. Tomasino writes: > 1. Listing things > 2. Quick instructions where headers are overkill > 3. Track listings > 4. Top 10 lists > 5. The same reasons we want bullets the reflow > 6. Recipes! I think Tomasino's response to Julien, in plain text, just proved the _lack_ of need for lists. Want a list? Write a list.... handled. I really really really do not see the point of adding all of this markup :-/ Just serve files as markdown if you want markdown and let clients either support markdown or not. I really do not want to, and likely will not, add support for lists in my current or any future gemini client I make. I don't see the point. I do appreciate all of the work and thought that has gone into trying to figure these things out. It just feels unnecessary for a simple protocol like gemini. Why get complex when just serving markdown or html or whatever you like will do the job and people can read it as they see fit. We are definitely starting to get into the territory with the proposals where some client authors will support some things and some wont... which will create a fractured view of gemini to users. So why not keep it simple and cohesive by just having links and move on to other things? Just my two cents at the moment. From solderpunk at SDF.ORG Sun Jan 19 20:01:23 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 19 Jan 2020 20:01:23 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <574762512.1104.1579461314537@ichabod.co-bxl> References: <574762512.1104.1579461314537@ichabod.co-bxl> Message-ID: <20200119200123.GA7786@SDF.ORG> On Sun, Jan 19, 2020 at 08:15:14PM +0100, Brian Evans wrote: > I've mostly been sitting back through all of this list and raw mode > talk, only occasionally popping in small points here or there. But > I figure if this is a community decision it is good if those in the community > voice their support or lack of support for ideas. It definitely is, if people who don't feel good about proposed changes keep quiet and all I hear on the list is agreement it will just lead to the most vocal members driving things. I'm glad you've spoken up. This will be a quick and kind of partial reply, sorry. But, very briefly, I understand the concern about lists (of all the recent proposed changes, they are the least valuable IMHO because they only concern presentation). How do you feel about the heading proposals? IMHO they are probably the most valuable of these changes because they can be totally ignored from the perspective of presentation and yet still do genuinely useful stuff, like automatically generated tables of contents, which do nothing but make large and well-structured text content easier to read, or help with automatic generation of user-friendly menus or feeds to bodies of content. That's good stuff, surely? > I really do not want to, and likely will not, add support for lists in my current > or any future gemini client I make. I don't see the point. As I intend to actually write all this stuff in the spec, if I decide to adopt it, supporting the list line types will be strictly optional, so this is absolutely fine. For a text-based client like Bombadillo, the difference between supporting lists and not is an extremely small aesthetic thing that some users probably won't even notice. > I do appreciate all of the work and thought that has gone into trying to > figure these things out. It just feels unnecessary for a simple protocol > like gemini. Why get complex when just serving markdown or html or > whatever you like will do the job and people can read it as they see fit. > We are definitely starting to get into the territory with the proposals > where some client authors will support some things and some wont... > which will create a fractured view of gemini to users. So why not keep it > simple and cohesive by just having links and move on to other things? The fractured Geminispace thing is a real concern, but actually I think an argument can be made in both ways here. If text/gemini is kept extremely simple and plain and the official policy is "Serve Markdown or HTML if you want even basic text styling" then people may well do that. Some clients will add support for rendering those but many won't because it's so much more complicated (far worse than what has been proposed here!). Then we end up with two regions of Geminispace, the text/gemini subspace which anybody can visit and the Markdown subspace which only people using the fanciest clients can visit. IMHO, this is a worse kind of fracturing than one where we adopt the proposed new text/gemini syntax and different clients implement different subsets of the optional features. After all, the only reason I've been so positive about these recently proposed changes is that they all seem to degrade very gracefully if a client doesn't recognise them and treats them equivalent to text. The degree of fracturing possible is actually very slight. > Just my two cents at the moment. One cent of opinion from client or server authors is valued equal to one dollar of opinion from anybody else. :) Cheers, Solderpunk From solderpunk at SDF.ORG Sun Jan 19 21:30:35 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 19 Jan 2020 21:30:35 +0000 Subject: What shall we call Gemini logs, anyway? In-Reply-To: <367CB83F-81E0-4E35-BAAF-B3C8D8EF63C8@typed-hole.org> References: <20200118173048.GG4518@SDF.ORG> <367CB83F-81E0-4E35-BAAF-B3C8D8EF63C8@typed-hole.org> Message-ID: <20200119213035.GB7786@SDF.ORG> On Sun, Jan 19, 2020 at 01:04:44AM +0100, Julien Blanchard wrote: > I used ?Flight Journal? for mine because gemlog reminds me too much of Ruby gems, but that?s not really something I would advise. glog sounds a bit weird too, at least when trying to pronounce it. > Looking at the Gemini program I found the word ?capsule?. What?s interesting is that in French it can mean ?a short column on a given subject? and I kinda like how it sounds. The "capsule" idea was proposed earlier on (I forget by who, sorry!). I kind of like it, but I've come to think of it as a slightly more general term, like "gopherhole" - i.e. it doesn't actually imply regular updates. I asked the Gemlog I know how he felt about us using "gemlog" in this context and he had no objections! Cheers, Solderpunk From solderpunk at SDF.ORG Sun Jan 19 22:18:26 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Sun, 19 Jan 2020 22:18:26 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200119200123.GA7786@SDF.ORG> References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> Message-ID: <20200119221826.GC7786@SDF.ORG> On Sun, Jan 19, 2020 at 08:01:23PM +0000, solderpunk wrote: > The fractured Geminispace thing is a real concern, but actually I think > an argument can be made in both ways here. If text/gemini is kept > extremely simple and plain and the official policy is "Serve Markdown or > HTML if you want even basic text styling" then people may well do that. > Some clients will add support for rendering those but many won't because > it's so much more complicated (far worse than what has been proposed > here!). Then we end up with two regions of Geminispace, the text/gemini > subspace which anybody can visit and the Markdown subspace which only > people using the fanciest clients can visit. IMHO, this is a worse kind > of fracturing than one where we adopt the proposed new text/gemini > syntax and different clients implement different subsets of the optional > features. After all, the only reason I've been so positive about these > recently proposed changes is that they all seem to degrade very > gracefully if a client doesn't recognise them and treats them equivalent > to text. The degree of fracturing possible is actually very slight. Oh, there was also the argument made at some point that if Gemini doesn't have any standardised syntax for these common formatting tasks, ambitious clients might start trying to recognise the most popular non-standard ways of doing it, which could easily lead to divergent implementations across clients. So, better to provide a standard way to do to provide us control and uniformity. I don't mean to claim either of these arguments are bulletproof, I just think that a principle of "we shouldn't do anything that risks fragmentation of Geminispace" (which is obviously a good principle) necessarily comes down clearly on one side of this question or the other. Cheers, Solderpunk From lazar.michael22 at gmail.com Mon Jan 20 02:01:17 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Sun, 19 Jan 2020 21:01:17 -0500 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200119221826.GC7786@SDF.ORG> References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> <20200119221826.GC7786@SDF.ORG> Message-ID: I think there have been a lot of good suggestions coming from all sides. Here's my take on a compromised format that tries to take everything into account while also inserting a few of my own opinions: Parser pseudo-code (actually, it's valid python): ``` preformat_mode = False preformat_buffer = [] for line in document: if line.startswith('```'): if not preformat_mode: # Start preformat block preformat_mode = True preformat_buffer = [] else: # End preformat block preformat_mode = False display_preformat_block(preformat_buffer) elif preformat_mode: # Inside of preformat block preformat_buffer.append(line) elif line.startswith('###'): display_header_level_3(line) elif line.startswith('##'): display_header_level_2(line) elif line.startswith('#'): display_header_level_1(line) elif line.startswith('=>'): display_link(line) elif line.startswith('---'): display_horizontal_rule() else: display_paragraph(line) if preformat_mode: # Flush the preformat block if there was no end tag add_preformat_block(preformat_buffer) ``` This pseudo-code was written with "fancy" gemini clients in mind. In other words, this should be close to the worst-case scenario for how complicated a gemini document parser would ever need to be. ## Preformat mode Many clients are going to want to display a preformat block of text in a horizontally scrollable window or some other type of block widget. This pseudo-code reflects that by sticking the pre-formatted lines in a separate buffer until the end of the block. I think this is a more accurate representation of what most client parsers would end up looking like. ## Headers I'm of the opinion that there should only be a fixed number of header levels. It keeps the matching logic flat and straightforward. Three levels is few enough that most clients should be able to come up with distinct styles to display them. Fixed header lines are trivial to parse and provide a lot of utility for organizing a document and linking to sub-sections. ### Ordered Lists & Unordered Lists Lists are tricky because while they would be nice to have, the complicate the parsing significantly. In order to parse a list while preserving its semantic structure, you will need to keep track of where it starts and ends. Nested lists complicate this even further, no matter which syntax for nesting is used. Parsing lists semantically would require keeping a separate buffer for each type of list, and then keeping flags and making sure that these buffers are flushed after the last element in the list. Because of this, I do not believe that they pass the power-to-weight ratio smell test. For authors, they still have a few choices for lists: 1. Stick the list in a preformat block 2. Write the list in plain mode without special formatting I accept that neither of these options is *ideal* for all use cases, but I think they are *good enough* for most use cases. Don't forget that unicode bullets can already be added directly in gemini documents if the author wishes to do so. ### Quotes Quote blocks with ">" would be ok if we could count on them being only a single line long. However, many quotes will necessarily include line breaks that should be displayed together in a single block. This complicates parsing in the same way that lists do, so I think that quotes should also be omitted for the same reason. If you want to display something like a quote from a mailing list message, I think that would be a perfect candidate for copying it into a preformat block. For other types of quotes, stick them between two horizontal rules to separate them from the surrounding text. ### Horizontal Rule I find the horizontal rule useful for separating sections of a page. I see them commonly use on gopher to add a footer to the bottom. They can likewise be used for header sections. ``` Header --- Content --- Footer ``` The following gemini sites already use some form of horizontal separator on their front pages (the precise syntax varies): - gemini://vger.cloud/ - gemini://gemini.conman.org/ - gemini://zaibatsu.circumlunar.space/ - gemini://carcosa.net/ - gemini://yam655.com/ I think that since horizontal rules are easy to parse and they add utility for structuring pages, they should be included in the spec. ### Other Random Opinions - Leading and trailing whitespace should be stripped from all of lines outside of the preformat block. If you're allowing a non-monospace font for these elements, then leading whitespace can look inconsistent and trailing whitespace serves no real purpose. By leading whitespace, I mean that ## heading text Should be interpreted as "heading text", not " heading text". - I have no opinion on whether the ``` should allow text after it on the same line. I think I would be satisfied enough with the above document to at least try it out by converting all of my existing gemini content. I also think I would be fine keeping everything fixed-width and hard wrapping. I *don't* think I would want to implement nested lists or quote blocks, or anything significantly more complicated than what is outlined above. - mozz From solderpunk at SDF.ORG Mon Jan 20 09:58:33 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Mon, 20 Jan 2020 09:58:33 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <574762512.1104.1579461314537@ichabod.co-bxl> References: <574762512.1104.1579461314537@ichabod.co-bxl> Message-ID: <20200120095833.GA13472@SDF.ORG> On Sun, Jan 19, 2020 at 08:15:14PM +0100, Brian Evans wrote: I've been thinking more about lists and, while I understand the concern about maintaining simplicity, I really do think that: > I think Tomasino's response to Julien, in plain text, just proved the _lack_ of > need for lists. > > Want a list? Write a list.... handled. is a bit too simplistic. Tomasino's response consisted entirely of short list items which didn't require wrapping, and also happened in a plain text email, which is a hard wrapping environment and so not comparable to a "long line" text/gemini document. In Gopherspace, it's very common for people writing lists to format multi-line list items "nicely", e.g. people write this: --- * To demonstrate endurance of humans and equipment in spaceflight for extended periods, at least eight days required for a Moon landing, to a maximum of two weeks * To effect rendezvous and docking with another vehicle, and to maneuver the combined spacecraft using the propulsion system of the target vehicle * To demonstrate Extra-Vehicular Activity (EVA), or space-"walks" outside the protection of the spacecraft, and to evaluate the astronauts' ability to perform tasks there * To perfect techniques of atmospheric reentry and touchdown at a pre-selected location on land --- In preference to this: --- * To demonstrate endurance of humans and equipment in spaceflight for extended periods, at least eight days required for a Moon landing, to a maximum of two weeks * To effect rendezvous and docking with another vehicle, and to maneuver the combined spacecraft using the propulsion system of the target vehicle * To demonstrate Extra-Vehicular Activity (EVA), or space-"walks" outside the protection of the spacecraft, and to evaluate the astronauts' ability to perform tasks there * To perfect techniques of atmospheric reentry and touchdown at a pre-selected location on land --- I had worried, before writing this, that perhaps I was a lone weirdo for caring about this, but didn't have to search too far to find examples of others doing it, e.g. gopher://zaibatsu.circumlunar.space:70/0/~tfurrows/phlog/2020-01-16_worldEndingStuff.txt Now, this nice list formatting is simple and easy in a hard-wrapping environment with fixed line width. In a long line environment where clients wrap the line to the appropriate length, and the appropriate length is different for everybody, it's impossible for a content author to ensure readers get the first, nicer style of list formatting. As a consequence of client wrapping, we get stuck with the second. For lists of short items, there is no difference. Some might argue that this really doesn't matter and isn't worth adding complicating features for. But the fact that people writing for Gopher take the time to do the nice formatting suggests people actually care about this. And I don't think it's purely a matter of vanity and wanting our content to simply look pretty - if that's what we wanted, we wouldn't be writing for gopher! I actually think the nicer list formatting makes for a genuinely better reading experience. The list is much more easily and immediately recognisable as a list and it's much easier for the eyes to parse out the separate list items. This really is a functional thing, I think. A small one, I grant you, but a real one. It would be a shame if Gemini content had to be in some ways harder to read than Gopher content, and ironic if that happened as a result of the decision to have clients wrap lines to fit the viewport neatly in order to ensure a better reading experience for everyone! After thinking about this I am more convinced than I was yesterday that having some defined markup for lists is worthwhile. That said, I'm becoming rapidly less convinced about *nested* lists, especially without a limit on the permitted level of nesting. More on that when I reply to Michael's recent email, which will probably be much later today. Cheers, Solderpunk From solderpunk at SDF.ORG Mon Jan 20 10:28:56 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Mon, 20 Jan 2020 10:28:56 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> <20200119221826.GC7786@SDF.ORG> Message-ID: <20200120102856.GA4819@SDF.ORG> Just a quick response for now: nice post, thanks, there's a lot in here that I agree with (and I had been starting to think similar things about quotes), but can I ask you to elaborate on: On Sun, Jan 19, 2020 at 09:01:17PM -0500, Michael Lazar wrote: > Lists are tricky because while they would be nice to have, the complicate the > parsing significantly. In order to parse a list while preserving its semantic > structure, you will need to keep track of where it starts and ends. Nested > lists complicate this even further, no matter which syntax for nesting is used. > > Parsing lists semantically would require keeping a separate buffer for each type > of list, and then keeping flags and making sure that these buffers are flushed > after the last element in the list. Because of this, I do not believe that they > pass the power-to-weight ratio smell test. In particular, what do you mean by "parsing lists semantically"? At no point in these discussions have I been envisaging anything to do with lists which requires clients to recognise or keep track of whether or not they are "inside" a list or not, or sticking lists in buffers. I have imagined list items standing alone and "lists" being an emergent property of a document that clients have no awareness of - in exactly the same way that "paragraphs" are an emergenty property of lines (if some of those lines happen to be blank). Well, that's true for unordered lists, at least. Ordered lists are another story Cheers, Solderpunk From tomasino at lavabit.com Mon Jan 20 12:09:01 2020 From: tomasino at lavabit.com (James Tomasino) Date: Mon, 20 Jan 2020 12:09:01 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200120102856.GA4819@SDF.ORG> References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> <20200119221826.GC7786@SDF.ORG> <20200120102856.GA4819@SDF.ORG> Message-ID: On 1/20/20 10:28 AM, solderpunk wrote: > Well, that's true for unordered lists, at least. Ordered lists are > another story Ordered lists are--so far--the only thing that really does break the linear line-by-line processing approach. If they are sacrificed to the gods of a cleaner spec, I for one wouldn't cry too much. Unordered lists that support reflow is the more important bit. One could always cheat and just use unordered lists and start each one with a number: * 1) item one! * 2) A really long item two that will be wonderfully reflowed on small screens by our awesome client writers. ** 2a) I'm lookin' at you, Michael! ** 2b) And the rest of you too. ;) Saves us a bit o' logic. Keeps everything line based. You could run a stateless renderer on a stream now and it wouldn't choke. From solderpunk at SDF.ORG Mon Jan 20 12:32:30 2020 From: solderpunk at SDF.ORG (solderpunk) Date: Mon, 20 Jan 2020 12:32:30 +0000 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> <20200119221826.GC7786@SDF.ORG> <20200120102856.GA4819@SDF.ORG> Message-ID: <20200120123230.GB16971@SDF.ORG> On Mon, Jan 20, 2020 at 12:09:01PM +0000, James Tomasino wrote: > One could always cheat > and just use unordered lists and start each one with a number: > > * 1) item one! > * 2) A really long item two that will be wonderfully reflowed on small > screens by our awesome client writers. > ** 2a) I'm lookin' at you, Michael! > ** 2b) And the rest of you too. ;) > > Saves us a bit o' logic. Keeps everything line based. You could run a > stateless renderer on a stream now and it wouldn't choke. This has two other advantages, too. First, when processed by a simple client which opts not to treat list items as anything special, the result is still obviously an ordered list. Under the + proposal, it will look just like an unordered list whose author chose a different bullet character for reasons of taste. Ambiguous degredation is not graceful degredation! Second, authors can unambiguously refer back to a list item in later writing. I can say "Tomasino used a winky face in item 2b above" and you can all go back and find 2b and confirm this. If I said that and you were using a simple client that just rendered + and ++, it would be up to you to mentally figure out which point was 2b. And even if you're using an advanced client that does render ordered lists, I might write "2b" but your fancy client might use Roman numerals for second level lists and print 2ii instead of 2b, and again the connection wouldn't be immediate. This could only be solved by tediously specifying that first level lists MUST use Arabic numerals, second level lists MUST use lowercase letters, third level lists MUST use Roman numerals, and so on and so on. And then what happens when somebody uses more than 26 second level list items and we run out of lowercase letters to use? A spec that can avoid all these problems will be exactly the kind of long, tedious, fiddly spec that I really don't want us to use, and which nobody will want to code to anyway. I'm starting to think we should either drop the ordered list idea, or at the very least strictly limit it to one level with no nesting. Cheers, Solderpunk From lazar.michael22 at gmail.com Mon Jan 20 17:30:14 2020 From: lazar.michael22 at gmail.com (Michael Lazar) Date: Mon, 20 Jan 2020 12:30:14 -0500 Subject: FW: Text reflow woes (or: I want bullets back!)y In-Reply-To: <20200120102856.GA4819@SDF.ORG> References: <574762512.1104.1579461314537@ichabod.co-bxl> <20200119200123.GA7786@SDF.ORG> <20200119221826.GC7786@SDF.ORG> <20200120102856.GA4819@SDF.ORG> Message-ID: On Mon, Jan 20, 2020 at 5:29 AM solderpunk wrote: > > Just a quick response for now: nice post, thanks, there's a lot in here > that I agree with (and I had been starting to think similar things about > quotes), but can I ask you to elaborate on: > > On Sun, Jan 19, 2020 at 09:01:17PM -0500, Michael Lazar wrote: > > > Lists are tricky because while they would be nice to have, the complicate the > > parsing significantly. In order to parse a list while preserving its semantic > > structure, you will need to keep track of where it starts and ends. Nested > > lists complicate this even further, no matter which syntax for nesting is used. > > > > Parsing lists semantically would require keeping a separate buffer for each type > > of list, and then keeping flags and making sure that these buffers are flushed > > after the last element in the list. Because of this, I do not believe that they > > pass the power-to-weight ratio smell test. > > In particular, what do you mean by "parsing lists semantically"? > > At no point in these discussions have I been envisaging anything to do > with lists which requires clients to recognise or keep track of whether > or not they are "inside" a list or not, or sticking lists in buffers. > I have imagined list items standing alone and "lists" being an emergent > property of a document that clients have no awareness of - in exactly > the same way that "paragraphs" are an emergenty property of lines (if > some of those lines happen to be blank). By "parsing lists semantically" I mean that if I build an AST, I want all of the list items grouped together inside of single list object. This is how I did it when I was playing around with markdown a while ago [0]. From my research this seems to be the common way to do it [1]. Sophisticated gemini clients could utilize this is a variety of ways. Maybe you want to add a little bit of extra whitespace surrounding the list. Or you want to make sure that the your display does not cut-off half way through the list. Or you want to support re-ordering list items alphabetically. I don't know, the sky is the limit. I'm willing to admit that HTML has perhaps tainted my thinking here, but it just feels *wrong* to me to have an
  • without the surrounding