World of Spectrum....

[1024MAK]

btw, I already told you that RMartins was recently banned (without proper explanation) and I almost cant believe you didnt notice it.
Maybe it should have been banned, but we dont know that, and we probably will not find out.
And why Vega+ should it be banned anyway?
5000 fans have invested their money in this project and I dont see why it's inappropriate to talk about it on the Spectrum related website.
Do you have any logical explanation, from independent point of view?

I did not realise that RMartins was recently banned. Sorry if you mentioned it in an earlier post, I must have missed that bit.

I can understand why Vega+ is banned. I don't agree with discussion being banned. However, if I was the owner/operator, admin or moderator of a Spectrum forum, a thread discussing the Vega+ would be subject to close moderation, simply because threads about it often end up going bad. Don't get me wrong, I have no problem with people who paid money coming on and talking about how unhappy they are with the situation. But it's all the other mud slinging that goes on that I don't like.

Yes, and I'm one of the people who now has less money in my pocket, but no Vega+ to show for it. It is a deeply disappointing state of affairs. And the occasional dangled "updates" are rather depressing (at least, for me).

Mark

[Nomad]

WOS was one of the first websites I found on the internet back in the 90s, come to think of it its the only one from the dialup days that I think is still in existence. Like most of you guys and girls the Spectrum was a huge part of my childhood, so finding such a active community in the early days that was bringing together so much information on the machine that fascinated me was a big plus. I can remember all my friends at college being green with envy when i walked in with a set of the WOS CD-Roms one day. I think that single event destroyed the grades of most of the computer science class that year. But it was worth it. To have thousands of games, utilities all in one place. When I remember how many years I had lusted after just a few titles and now there were all there waiting to be used. It was great.

But the biggest asset was the forums, being able to research a topic and get the insight and experience of guys who in some cases are no longer alive or active in the community. Being able to find out answers to questions that were often times either never covered or poorly explained in the magazines or the books. It was a godsend. I think that is one of the biggest dangers is loosing all of that data. Not saying there are not competent coders still around but its much easier to search a forum for answers than bothering the same guys again and again with questions that have been answered elsewhere.

After that, the books and magazines were a way i wasted hours looking through. To be able to read books and magazines I didn't have the pocket money for as a kid was great. But with archive.org having a fairly comprehensive collection now its not such a bit worry as say the forum data from WOS.

I just hope the preservation scans, all the maps and other content that so many people devoted so much time to can be preserved and put to good use in some way in the future.

I guess even though I was not an active member of the site, there are many fun memories associated with stuff that I was able to do because it existed. Still I am optimistic about the future of spectrum coding, I see lots of interesting projects and if youtube is anything to go by there still seems to be a lot of interest in retro computing.

I mean things could be worse - look at what happened to the BBC Micro archive websites... (shudders).

[Einar Saukas]

We all share the same concerns about WoS decline and all the valuable information it contains. That's exactly what we are all working to prevent!

Right now, there's a copy of all files from the original WoS archive preserved at Archive.org. The entire content of Martijn's Infoseek database is stored inside ZXDB, with corrections and improvements. Both SpectrumComputing and ZXInfo provide basically the same information you can find at the original WoS archive, except SpectrumComputing currently adopts a more strict policy to only distribute permitted files. Even so, it recently got green light to redistribute all games from Hewson, Zenobi and a few others, more will certainly follow. And SpectrumComputing already achieved the same "spirit" of the old WoS forum from its best days.

The only part we still risk loosing, if WoS goes down, is all the forum history. Although WoS forum searches are still broken, thus in practice most of its content is already inaccessible anyway... Also there's a good chance Martijn kept a backup when he left, so if the forum disappears, we could simply contact him and ask for his approval to restore a read-only copy somewhere.

[fogartylee]

Correction. The entire WoS archive isn't available anywhere else. It's currently around 250gb, with 1tb still to process - around 1/4 of that will be used.

I'd be grateful if you'd stop fueling rumours that WoS is going anywhere thank you.

As for the forums - the search works as well as it can with 1m comments and the size of the database makes it very difficult to host anywhere but a dedicated service. It uses Sphinx as the search service which is about as good as it gets. Without Sphinx it takes 30-60 seconds to do a full search of the forums - with it, it takes less than a second.

You'd also need to check the legality of taking someone elses forum to host wherever you feel like it. A lot of people have posted on WoS over the years not expecting comments to be hosted elsewhere.

[Einar Saukas]

Hi Lee! I wasn't sure if you were reading this forum!

Correction. The entire WoS archive isn't available anywhere else. It's currently around 250gb

... that's mirrored at Archive.org.

with 1tb still to process - around 1/4 of that will be used.

Those 1Tb of unprocessed files, that were not added to WoS archive yet, are not part of the WoS archive mirror. Obviously.

I'd be grateful if you'd stop fueling rumours that WoS is going anywhere thank you.

I'm not fueling any rumors. I never said WoS will go somewhere, instead I explicitly wrote "if WoS goes down" and "if the forum disappears". Therefore I was discussing a possibility. This possibility would happen, for instance, in case you decided to leave WoS without a plan to replace you. And this has already happened once.

As for the forums - the search works as well as it can with 1m comments and the size of the database makes it very difficult to host anywhere but a dedicated service. It uses Sphinx as the search service which is about as good as it gets. Without Sphinx it takes 30-60 seconds to do a full search of the forums - with it, it takes less than a second.

I wasn't debating the reasons why WoS forum search is broken. I just mentioned that it is. I didn't even mention that Martijn's WoS forum (before it was migrated) already had a similar large volume and searches worked just fine.

You'd also need to check the legality of taking someone elses forum to host wherever you feel like it. A lot of people have posted on WoS over the years not expecting comments to be hosted elsewhere.

Yet another reason to contact Martijn about it and ask for his approval, as I suggested. And that's only as last resort, "if the forum disappears" as I wrote!

[fogartylee]

Not sure how often to keep repeating. Archive.org DOES NOT HAVE THE FULL ARCHIVE. Not even close to full. The archive is a lot more than was publicly available - we are addressing that along with the new data.

[Einar Saukas]

Not sure how often to keep repeating. Archive.org DOES NOT HAVE THE FULL ARCHIVE. Not even close to full. The archive is a lot more than was publicly available - we are addressing that along with the new data.

Semantics. You consider 1Tb of unprocessed files as "hidden part of WoS archive". I consider it as "not added to WoS archive yet". It's the same thing, under different names.

[fogartylee]

I wasn't talking about the extra 1tb. I was talking about the existing archive.

[Einar Saukas]

I wasn't talking about the extra 1tb. I was talking about the existing archive.

Interesting. Do you have any examples?

[fogartylee]

There's videos, loads of additional files for publishers, people, etc., the denied titles, books, denied magazines, hi-res inlays & numerous other documents.

[grahamken]

Hmmm...that would be an ecumenical matter!

[Ralf]

Hello Lee. I suppose I should say "welcome to the forums".

As for "hidden" WOS stuff I would say that if it can't be seen at the website, it's not part of the website. It's just unsorted stuff on your hard drive. But yes, we are discussing language here.

The problem is that all this stuff isn't secure in your hands. At any moment you can get angry, annoyed, tired or whatever and dissapear forever from the community together with all these assets. We know each other enough that I can consider it probable. So don't be surprised that people are making backups, as complete as currently possible.

[fogartylee]

There's also the slight issue that at least 1 publisher will take legal action should they see their books in the archive. Which is why the new archive won't have them... Permissions change.

[ADJB]

That's OK then, that will be a similar but incomplete copy of what I have.

A few people who had full access were advised to take copies and did so. That will include the directory's you have never seen or had access to on Martijns server like, for example, my personal set of folders which held work in progress and data awaiting processing.

If you do have these when can we expect the publication of the OCRed version of the weekly mags? A text version would be much easier to search.

[fogartylee]

Yes we do. The new site will let people choose between text versions and images (where text has been done), and additionally the text versions are to be a part of infoseek. We aren't loading new data until we switch over in the new year as we are replacing the mags with hi-res too.

[emook]

To be honest the new site sucks Lee. It looks awful and seems overly engineered for what it needs to be.

On mobile its even worse. Personally, I don't want to use that as my new WoS but then how would you know that as you never involved the community.

Hidden stuff on WoS? meh. When the FTP was open there was a lot of stuff that wasn't of interest. We we care about has already been taken care of

I'd much prefer have Einar involved in curating a new database, he's shown to be intellectually capable of taking on such a task.

Merry Christmas by the way.

[1024MAK]

We all share the same concerns about WoS decline and all the valuable information it contains.

Err, you have Martijn to thank for the decline of WoS. When he found true love, he more or less abandoned updates on WoS. And then after the site trundled on for a number of years, the server became unreliable, before finally failing completely.

That's exactly what we are all working to prevent!

Funny, I thought your objective was simply to maintain a database.

Right now, there's a copy of all files from the original WoS archive preserved at Archive.org. The entire content of Martijn's Infoseek database is stored inside ZXDB, with corrections and improvements. Both SpectrumComputing and ZXInfo provide basically the same information you can find at the original WoS archive, except SpectrumComputing currently adopts a more strict policy to only distribute permitted files. Even so, it recently got green light to redistribute all games from Hewson, Zenobi and a few others, more will certainly follow. And SpectrumComputing already achieved the same "spirit" of the old WoS forum from its best days.

First off, it's rare that Archive.org captures all publicity available data. Second, it's even less likely to capture non-publicly available data, or get pages/data if there are broken links. So when did Archive.org crawl WoS? When WoS was not being maintained by Martijn or more recently? Please also note that Archive.org is just as venerable to server or hard drive failure as any other site. Earlier this year, I could not access a site archived on Archive.org due to a system failure. I don't know if this was a temporary failure or something else.
SpectrumComputing only has a fraction of copyright holders permissions compared to WoS. And in case anyone had not noticed, the World of Spectrum site is online and no less functional than when this forum came online. Lee has recently posted on WoS his plans for the near future.

Spoiler

As we edge closer to the new site switching over, we are looking at additional features, storage and the future of WoS.

At the moment, the site takes up nearly 250gb - including the database size (approximately 2.5gb). This doesn't include books.

So, moving forward, we will be adding books next - these will be similar to the magazines pages, followed by software, hardware, and the rest of the old WoS data.

2018 will see us integrating the forums more into WoS. This means users will log onto WoS in general, and allow users to vote for software, comment on each database entry, as well as other goodies associated with personal accounts. We will also be integrating the majority of the forums into Infoseek. Add to this the language translations, text versions of scans and other work we are preparing for, there is a lot going on!

Finally, we are looking at lifting download restrictions as per old WoS.

So if anything, the WoS sites are expanding and improving. And the WoS forums look plenty active to me. The magazines section is looking really good now.

The only part we still risk loosing, if WoS goes down, is all the forum history. Although WoS forum searches are still broken, thus in practice most of its content is already inaccessible anyway... Also there's a good chance Martijn kept a backup when he left, so if the forum disappears, we could simply contact him and ask for his approval to restore a read-only copy somewhere.

The search system is not perfect, granted. But it is possible to find lots of stuff. So it's not as useless as you imply. Any possible archive of the forum data will be just as hard to use unless a pre-search database is constructed, or you throw lots of resources at it.

The forum software that this forum is using also has problems with it's search function. As this forum is still relatively small, it's not likely to be a problem at the moment. But should it grow to anywhere near the size of WoS forums or StarDot forums or any similar sized forum, then various problems may appear with the search functionality.

Given what happened, I think it unlikely that Martijn will have anything other than a very old backup. So even if he does have such a thing, it will be of limited value.

If everyone really values the online archive and Sinclair computer internet presence, then you should support all web sites that are about the Sinclair computers (ZX80, ZX81, ZX Spectrum etc) including WoS.

The infighting that has happened in recent times helps no one. So let's see some community spirit and goodwill for the New Year

And there are a number of other Sinclair computers web sites that would love more support

Mark

[Whacker]

Personally, myself like many of the casual spectrum users that have come back to the scene don't really care about promises of what may be uploaded in the future.. more what is available publically for all.

One BIG concern I have over WoS right now, and I see this echoed from many sources recently is that all resources, be it manuals, schematics, publisher history, etc. is that it's being placed in a manner where there is a notable sense of gatekeeper access, trying to wall off information into a closed environment where, if you're deemed unworthy / argue against the current viewpoint, a simple IP block or account ban will remove your visibility for information.

If the community is to thrive, we need to view all information that has been given to the scene as open to all, and maintain a unified opensource database. This is how creativity that leads to projects like the 'Next', Zaxxon's work and others occurs. By your own statement Lee you highlight that archive.org doesn't have a full backup of the site (albeit quiet a recent one), which worries me that there is a greater risk of lost documentation by hoarding between a few select individuals.

I loved the old WoS just as I love Spectrumcomputing now. It's not about how 'flashy' the site is or the bells and whistles that power the site in the background. It's about the ease of access and availability to all the information can be retrieved, it doesn't matter who has the biggest archive of data or how high res the images are. It's about enabling likeminded people to connect and collaborate ideas, build upon them in a open discussion and take the aging machine into another decade.

The moment any site becomes about one person's ego controlling the way it moves forward, the sites days are numbered (which is clear from the amount of users pulling backups)..

[1024MAK]

One BIG concern I have over WoS right now, and I see this echoed from many sources recently is that all resources, be it manuals, schematics, publisher history, etc. is that it's being placed in a manner where there is a notable sense of gatekeeper access, trying to wall off information into a closed environment where, if you're deemed unworthy / argue against the current viewpoint, a simple IP block or account ban will remove your visibility for information.

What gatekeeper access are you on about?

Mark

[balford]

I presume Whacker's talking about the requirement for individual API keys for programmatic access to WoS resources.

[Whacker]

permanent site logins to access all material,m (API keys), and of course IP banning is a good one...

I registered late november to find my IP was blocked 24hrs after (without a first post made) to the live.worldofspectrum.co.uk subdomain, one that's still inforce as of today. Pity really..

Of course if this is just a server hiccup, and nothing to do with my association to a Mr Andrews or Mr Cropper then I would withdraw my concerns...

[fogartylee]

Lovely to see peoples opinions here.

Just for clarity, when I put the mags back on, I was slated because the page load time was too long with an issue being an average of 70mb. So, I reduced the thumbnails and made browsing quicker and easier. On SC, the thumbnails are actually the full size images so still 70mb page size. I see no complaints about that.

But for further clarity, I also offered to write the javascript needed to access the WoS magazines API and display the thumbnails correctly, and ultimately by default have access to the hi-res pages without needing any additional work. Strangely that offer was ignored by the team here with no explanation. A hell of a lot of work has gone into the mags as they were never initially indexed, and there were pages missing, incorrect, etc - that has all been done (I do know there are still some problems we are looking at)

And for those that don't like how the new site looks? That's the whole point of the data and files being accessible to everyone. If you don't like the site, use the API and create your own!

[balford]

It is nice to see people's opinions here, and even nicer that people are allowed to express them freely

[fogartylee]

permanent site logins to access all material,m (API keys), and of course IP banning is a good one...

I registered late november to find my IP was blocked 24hrs after (without a first post made) to the live.worldofspectrum.co.uk subdomain, one that's still inforce as of today. Pity really..

Of course if this is just a server hiccup, and nothing to do with my association to a Mr Andrews or Mr Cropper then I would withdraw my concerns...

There is a bug in the anti-leeching script. Nobody is banned by IP unless they leech (and the bug will be fixed this evening).

Where have you got the idea you need a site login to access data? The forums are separate entirely to the rest of the site, so there's no way you've registered on the "live" domain because no option exists.

[Mike Davies]

The only part we still risk loosing, if WoS goes down, is all the forum history.

I have that covered: the whole publicly-visible forum as an archive of JSON documents. If WoS does then disappear, we can curate the useful information out of it and make it Google-searchable.