Dcrdata Versions Save

Decred block explorer, with packages and apps for data collection and storage. Written in Go.

v4.0.0

5 years ago

IMPORTANT NOTES

  1. This version of dcrdata requires a build of dcrd running JSON RPC server version 5.1.x, such as the dcrd v1.4.0 release.
  2. When upgrading from dcrdata v3.x, the PostgreSQL database does NOT need to be rebuilt from scratch. However, if upgrading from v2.x.y, a complete DB rebuild is needed.

Summary (work in progress!)

This is the release of dcrdata v4.0.0. Since the last published non-patch release (v3.1.0), the following changes were made:

Explorer pages

  • Complete redesign with new page styles, more information, and better organization.
  • Politeia Proposal(s) pages.
  • Listing aggregate block information on time intervals (week, month, etc.) on Block page.
  • Charts may use log or linear y axis scaling.
  • Exchange rate monitoring for Decred-to-fiat conversions.
  • lots...

HTTP APIs

  • Added /block/.../raw and /block/.../header/raw endpoints. These return the hexadecimal encoded serialized bytes of the block or block header.
  • Added /agendas and /agenda/{agendaid} endpoints.
  • "hex" JSON field added to several "raw" endpoints (TODO: list endpoints here)
  • "network_name" added to API Status response
  • Compression is optional. Use --compress-api to enable compression in dcrdata's response handlers.
  • more...

Nuts and Bolts

The following changes are of most interest to site operators and developers:

  • The dcrdata module is now v4.
  • New build process. Webpack via npm. See README.md and dev/build.sh to learn more.
  • Support for go 1.11.5 and 1.12.1.
  • Support for dcrd v1.4.0 (JSON-RPC v5.1.0).
  • New config settings (see --help for details):
    • API: insight-limit-rps, max-api-addrs, compress-api
    • DB: purge-n-blocks, fast-sqlite-purge, pgtimeout, hidepgconfig, addr-cache-cap
    • Politeia: proposalsdbfile, politeiaurl
    • Exchange bot: exchange-monitor, disable-exchange, exchange-currency, ratemaster, ratecert
    • Site config: mainnet-link, testnet-link
  • Removed config settings: mempool (it is required), dumpallmptix (was a debug tool)
  • Updated CONTRIBUTING.md, and moved it to the docs folder.
  • Information from dcrd's new getblockchaininfo RPC is used in many places.
  • dcrd's notification sequence for chain reorganization events has changed with RPC v5.0.0, and dcrdata was changed to work with the new sequence. JSON-RPC server versions older than v5.0.0 are not compatible. Use the current dcrd release.
  • New "gov" packages (gov/agendas and gov/politeia), and the /proposals page.
  • New pubsub package, including related packages (pubsub/psclient and pubsub/types) and a demo client app (pubsub/democlient). pubsub's main types are PubSubHub and WebsocketHub. PubSubHub manages the collection and distribution of block chain and mempool data. WebsocketHub and its event loop manage all websocket client connections. psclient defines a Client type that can help writing apps that connect to a dcrdata pub-sub server.
  • mempool package has expanded functionality as a central hub for mempool data. explorer and pubsub now get their mempool updates from the mempool package's MempoolMonitor type. explorerUI and PubSubHub implement the MempoolDataSaver interface.
  • New exchanges package that implements an exchange rate data collection bot.
  • New dcrrates package and rateserver app for running a remote exchange data bot with one ore more dcrdata instances as clients.
  • New testutil/apiload package for load testing the dcrdata APIs.
  • New db/cache package for caching address data. Presently used in dcrpg.
  • The chainMonitor in the blockdata package replaced BlockConnectedSync with ConnectBlock. It is no longer necessary to run the BlockConnectedHandler goroutine.
  • Added the ability to purge N blocks on startup to assist in recovery.
  • dcrpg (dcrdata's PostgreSQL interface) changes:
    • dcrpg table versions are now 3.10.0 (dcrdata v3.1 had tables at version 3.5.5).
    • ChainDB queries have a query timeout now, and a cancelable context.Context.
    • Multiple caches added for address (db/cache) and ticket pool data.
    • When the log level is debug or verbose, the PostgreSQL server (postgres) settings are displayed in a table on dcrdata startup to assist in debugging configuration issues.
    • Time in PostgreSQL tables is now stored as TIMESTAMPTZ instead of INT64. Time fields of Go structs in api/types are changed to the new TimeDef type. Tables: addresses, vins, blocks, transactions.
    • chainwork column (TEXT) ADDED to the blocks table.
    • extra_data, merkle_root, stake_root, and final_state columns are REMOVED from the blocks table.
    • The agendas table scheme is completely new. The old table is dropped and recreated from scratch.
    • New agenda_votes table.
    • block_time column (TIMESTAMPTZ) ADDED to the votes table.
    • New testing table.
    • dcrdata sets the timezone to UTC at the beginning of each PostgreSQL session.
    • dcrpg now uses dcrjson/v2 and rpcclient/v2 modules.
  • dcrsqlite changes:
    • Now exports the main SQLite interface type, WiredDB.
    • New block data cache.
    • WiredDB will request a shutdown with a provided function if the "database is locked" error is encountered on a table write operation.
    • UnconfirmedTxnsForAddress is moved from dcrsqlite to rpcutils.
    • Side chains are now tracked: hash is the new primary key. The is_mainchain and is_valid columns are added. Ability to import side chain data from dcrd also added.
    • explorer.New now takes a config struct, ExplorerConfig.
  • Dependency changes:
    • Updated direct deps: asdine/storm, caarlos0/env, all dcrd modules, dcrwallet/wallet, dgraph-io/badger, didip/tollbooth, go-chi/chi, google/gops, googolleego-socket.io, mattn/go-sqlite3, rs/cors, sirupsen/logrus, golang.org/x/net,
    • Added deps: politeia, golang/protobuf, google.golang.org/grpc, jessevdk/go-flags
    • Removed deps: btcsuite/go-flags, oleide/lane

Contributors

Summarized by number of commits:

     3  Ademu Enyo Anthony
    76  buck54321
    23  David Hill
     1  Jamie Holdstock
   160  Jonathan Chappelow
     2  Macaulay Davies
    28  Migwi Ndung'u
     2  Wisdom Arerosuoghene
     23 ZeroASIC
     1  degeri
    25  gozart1
     1  jujuhar

394 changed files with 47,963 additions and 19,205 deletions (yes, that is enormous!): https://github.com/decred/dcrdata/compare/v3.1.0...v4.0.0

v3.1.2

5 years ago

This is patch release 2 of the dcrdata 3.1 series.

Fixes:

v3.1.1

5 years ago

This is patch release 1 of the dcrdata 3.1 series.

Fixes:

  • mempool last block JS age controller fixed (86464624da453aab67ea2f266cd7fad4ef43c41d)
  • address go-sqlite "database is locked" errors (6a9fed5ddc25877ae72c64fca0f0d46e150738b1). This can slow down sqlite operations since it limits DB "connections" to 1. In 4.0, the limit is removed since the go-sqlite version dependency is updated to a version built with correct multi-threading flags.
  • resolve a possible panic on shutdown (261048e4ee9199be28733f9ab39f4a77f57c477a)

v3.1.0

5 years ago

IMPORTANT NOTES:

  1. This version of dcrdata requires a build of dcrd running JSON RPC server version 5.0.x, such as the current development version of dcrd v1.4.0.
  2. When upgrading from v3.0.x, the PostgreSQL database does NOT need to be rebuilt from scratch. However, if upgrading from v2.x.y, a complete DB rebuild is needed.

This is the release of dcrdata v3.1.0. Since the last published non-patch release (v3.0.0), the following changes were made:

  • Build with Go 1.11 modules, defining dcrdata as v3 module. (2f68351, 5f24e74f, 789129d, 44406ef6, 92ff8a23)

  • Support only Go v1.11. (70ed11e, 3beacda)

  • Major initial sync performance improvements. (d01def5a3, 636cfa5b)

  • Add netparams package for access to network service port numbers.

  • New version scheme with "appPrerelease" and "appBuild" info to generate version strings like v3.1.0-release or v3.1.1-pre+dev. (cc0e85c) This matches the version scheme now in use by dcrd and dcrwallet. The build syntax for dcrdata is as follows:

    go build -ldflags "-X github.com/decred/dcrdata/v3/version.appPreRelease= -X github.com/decred/dcrdata/v3/version.appBuild=`git rev-parse --short HEAD`"
    
  • Bugs fixed on /agendas page (e54cfa10) and with agendadb (4b074b3)

  • Improvements to /address page charts, ensuring the group-by interval selected is the most meaningful for the current address. (d105024e)

  • Add the ability to use environment variables for configuration. (f2903f1a)

  • All votes, including side chain votes, are included on the mempool page. Votes on blocks other than the best block are highlighted in red. (93b5788)

  • Votes on the mempool page are given an index according to which ticket is spent. (8a2e2ad3 and b4ae964d)

  • Basic Dockerfiles. (61a38fb)

  • Restore next expected ticket price and bounds, like we used to show it. (ba0f9ee)

  • Support the /sync Insight API endpoint. (aac0885c)

  • Block page now handles "future" blocks with height not yet reached. (71c1d03)

  • Side chain support:

    • Queries and /side page (b159166, 706b8aad). /side is a list of side chain blocks with links to them, their parents, and their children, if any. The /block page is updated to indicate if the block is a side chain block, and if it is approved by stakeholders.
    • Shows on the transaction page a list of blocks containing that transaction. There should be only one block that is both mainchain and stake-validated. Any other blocks will be "Mainchain: false" or "Valid: false" or both. When a transaction is found in multiple blocks, list them all with unambiguous links by hash. Also pass the transaction index via ?ind= URL parameter so that we can use it on the block page to highlight the transaction. (091b91b2, 412dd7ef)
  • Retrieve and import side chain blocks from dcrd via the new getchaintips RPC. (254ecab6, 7adf9bf2, 235ddd8d)

  • Stake-invalidated blocks page at /rejects (19953f66), and improved stake-invalidation handling. (0dfef89d)

  • Invalidated transaction support. If a transaction is not known to dcrd, it is because it has been orphaned by a reorg, invalidated via stakeholder disapproval, or both. dcrd will now attempt to get transaction details from the dcrpg database, which may have decoded and recorded the transaction. A message is shown on the transaction page when the transaction is not found in any valid and mainchain block. (2cfeccd2)

  • Special handling of genesis block coinbase transaction, which is invalid. (10fccfc6) Show an attention symbol in red next to the tx hash. A title (hover text) is shown explaining why it is special. The link points to code (chaincfg/genesis.go).

  • Concurrency-related fixes in explorer package. (3ae6c99e)

  • Style invalidated transactions on the block and transactions pages: gray out and change background color of regular transaction table data, including the coinbase, on a block's page. On the blocks list page, indicate invalidated blocks. (8d924bf8)

  • Improved error handling on /address page. (700d3c50)

  • Address API no longer returns null for address with no transactions, but an object with an empty slice of transactions. (979872d3)

  • Accelerate /ticketpool page. (974f78fb, c2160667)

  • Insight API fixes (34e508d, 36e060db)

  • Insight API docs (3db503d0, 6580ff2e)

  • Address page and query performance improvements. (3e42cd1, 08147c3)

  • Patch valid_mainchain column of addresses table. (078483d7)

  • Fix the address page for addresses with only unconfirmed transactions. (38c51a14)

  • Input/output highlighting on the address and transaction pages. To support the highlighting, a new route for identifying the input or output of interest is added: /tx/{txid}/{in|out}/{idx}. The input/output that pairs with the corresponding outputs/inputs in a transaction's view are obtained via queries into the vins table. (60d033d4)

  • Various sync and startup improvements and fixes. (08f1839f, 97ed3bd1, 769abd21, 59c6fbd8)

  • dcrpg query maintenance/reorganization. (102df3e5)

  • Add the ability to show transaction output spending information via ?spends=true on the following API endpoints: /api/tx, /api/tx/decoded, and /api/txs. (e96c9d62)

  • Prototyping new home page at /nexthome. (694eaee5)

  • Add a syncing status page that starts up before dcrdata completes syncing with dcrd. (44c8a086, eab8e434)

  • Security improvements. (5e473a85, 28c3d697)

  • Adjust the config/log/data paths if appdata is set. (98b418b2)

  • Show mining fees on /block page. (3a97e639)

  • Docs and sample config updates. (b97a023d, f6d2f423, eec01acf, 28c3d697)

  • Add new page /ticketpricewindows that lists all of the historical ticket price windows. (e8ceddea, 7461f90a)

  • Replace the shared quit channel with a context.Context from the "context" standard library. (71812ded)

  • Show Tx Type in the address page's tx table. (052e344c)

  • Overhaul reorg handling for dcrd's JSON RPC v5.0.0 notification sequence. (2fd7d047)

  • Improved support for browsers with JavaScript disabled. (a66fba37)

  • Better address validation. (dde716d8)

  • Numerous UI, formatting, and wording tweaks. Many other minor and under-the-hood changes.

16 code contributors (alphabetical): @ademuanthony, @buck54321, @chappjc, @codemaestro64, @degeri, @dmigwi, @gozart1, @itswisdomagain, @jholdstock, @Jujhar, @logicminds, @McEdward, @papacarp, @RogueElement, @s-ben, @SeaLightHouse.

158 changed files with 16,122 additions and 6,485 deletions: https://github.com/decred/dcrdata/compare/v3.0.0...v3.1.0

v3.0.3

5 years ago

This is a patch release with one fix since v3.0.2:

When appdata is set, dcrdata now adjusts the config/log/data paths.

With appdata set to a non-default value via either a command line switch or the environment variable, this updates the default config file location to avoid an error. A missing config file in the default appdata directory should not have been an error.

This change also updates the default config file location when a custom appdata is set so there is no error if the config file is missing. It is only an error if the config file location is explicitly set and does not exist.

A small optimization is also made such that the API status is not updated with an invalid height when DB is empty. When first starting with an empty sqlite DB, do not send a bogus height (4294967295 = uint32(-1)) on the API status update channel.

The block height is added to the error message if no block is returned by appContext.BlockData.GetBestBlockSummary when performing this lookup in response to an API status update signal received by appContext.StatusNtfnHandler.

https://github.com/decred/dcrdata/compare/v3.0.2...v3.0.3

v3.0.2

5 years ago

This is a patch release with several important fixes. The postgresq DB table format is bumped to 3.5.4, requiring a short automatic upgrade on initial startup of this version.

The following changes were made since v3.0.0:

https://github.com/decred/dcrdata/compare/v3.0.0...v3.0.2

v3.0.0

5 years ago

NOTE: The PostgreSQL database must be rebuilt from scratch (using the dcrdata executable) as the scheme is incompatible with previous versions and an upgrade was not practical.

This is the release of dcrdata v3.0.0. Since the last published release (v2.1.3), the following changes were made:

  • New "Merged Debits" transaction filter on an addresses page, which eliminates display of multiple inputs belonging to the same transaction, instead consolidating to a single row for such a transaction. This greatly improves display of the project fund spending activity.
  • The addresses table has a new scheme, which supports various improvements to the address web page.
  • Side chain tracking, supporting many new upcoming features related to reorganization and stake approval.
  • Cool new historical charts for the network, and on the address page
  • Initial experimental Insight API support (HTTP via /insight/api, and websocket/socket.io via /insight/socket.io).
  • Stake agenda DB table and web page.
  • Add block rewards breakdown API endpoint /block/.../subsidy
  • Distinct color themes for the networks: mainnet, testnet, simnet
  • Go module support (github.com/decred/dcrdata/v3)
  • Mempool page improvements with respect to old and/or side chain vote display and styling.
  • Bring back the next expected ticket price display on the home page.
  • testnet3 support (no testnet2)
  • Go 1.11 support, with module build support.
  • Add --no-dev-prefetch option to disable computing the project fund balance after each new block.
  • Add --sync-and-quit option to immediately quit after syncing to the network best block.
  • Replace error page with a more informative status page.
  • Improved display of both unconfirmed and confirmed transactions on the address page.
  • New dcrsqlite tests and testutil package.
  • Special handling for dummy sstxchange address

9 code contributors (alphabedical): @chappjc, @codemaestro64, @dmigwi, @gozart1, @JFixby, @papacarp, @RogueElement, @SeaLightHouse, @snadorp. 106 changed files with 9,064 additions and 1,822 deletions:

https://github.com/decred/dcrdata/compare/v2.1...v3.0.0

v2.1.3

5 years ago

NOTE: Not compatible with the data tables scheme or data directory layout of v1.x, v3.x, OR master. NOTE 2: This release supports testnet3, not testnet2. The latest release with testnet2 support is v2.1.0.

This is the release of dcrdata v2.1.3. Since the last published release (v2.0), the following changes were made:

  • Filtering address transactions by credit/debit
  • Added /api/address/{addr}/totals API endpoint
  • Added chain parameters page
  • Fresh hamburger menu with animated transitions
  • Keyboard navigation
  • Ticket return calculations shown on home page
  • Initial Insight API framework.
  • http server has conservative read/write timeouts
  • Pagination to /api/address endpoints
  • Added batch transactions request API endpoints /txs and /txs/trimmed
  • Begin refactoring JavaScript to use Stimulus.js
  • Websocket robustness improvements and bug fixes
  • Documentation for Go types used in API responses
  • Improved API status endpoint with db block height
  • Added coin supply API endpoint /api/supply
  • Improve PostgreSQL tuning instructions.
  • gops agent and instrumentation options
  • Dev fund is now "project fund"
  • Miner fees displayed for coinbase transaction
  • Performance improvements in stakedb package.
  • Ticket pool database migrated from storm to badger.
  • Many bug fixes and performance improvements.

14 code contributors: @gozart1 , @samparsky, @McEdward, @girino, @papacarp, @RogueElement, @jrick, @JFixby, @orthomind, @lte13, @timi-ola, @SeaLightHouse, @snadorp, @chappjc.

113 changed files with 9,100 additions and 2,222 deletions.

https://github.com/decred/dcrdata/compare/v2.0...v2.1.3

v2.0

5 years ago

NOTE: Not compatible with the data tables scheme or data directory layout of v1.3 OR master.

This is the release of dcrdata v2.0. Since the last release (v1.3.0), the following changes were made:

  • Desktop notifications (optional) for new blocks. Just click the green websocket light at the bottom right and click "allow"
  • Development fund "balance" is updated asynchronously to prevent slow block processing.
  • "Lite" mode is now the default. To start with PostgreSQL in full mode, use --pg.
  • More robust handling of ticket DB, with better crash recovery.
  • Support go1.10, drop go1.8
  • Lots more

6 contributors: @gozart1 , @RogueElement , @McEdward , @dajohi , @papacarp , @chappjc, @raedah . 6,722 additions and 1,701 deletions. https://github.com/decred/dcrdata/compare/v1.3.1...v2.0

v1.3.2.0.0.0.0.0.0.1

6 years ago

This is the Hcash edition of dcrdata. It works (probably)!