You are on page 1of 6

Assessment

1) Technology
Software development team:
• Overview, org chart, and bio/background for key members of technical team
o Technical management, Architect(s), Key individual contributors
• What is the relationship between your product management team, customers, and your software
development team?

Software development process:


• Describe your process for specifying, designing, and building software
• What have you done to make this process predictable and repeatable?
• What role does prototyping and user-testing play in the design and dev process?
• What is your incentive structure for your development staff?
• What are the key metrics you use to determine how effective your software development process is?
o How have you performed against those metrics for the past 18 months?
o Can you quantify and support that performance?

QA process:
• Describe and specify your QA process and organizational structure
• How many QA engineers do you have?
• What are the key QA metrics you keep?
o Are they quantitative or qualitative?
• How have you performed against those metrics for the past 18 months
• How do you handle regression testing?
o Is it automated?
o Explain the process and tools used
• How do you test for multiple deployment and end-user configurations?
• How do you test scalability, availability, security, etc.?

Software technology:
• Describe your overall software architecture
o What are the key components of your system?
o What are the key design rules and patterns that describe how those components interact?
• How have you designed fault tolerance and high availability into your system?
• What is your scaling strategy (for both hosted and shrink-wrapped scenarios)?
• How do you handle authentication, access control, and encryption?
• What do you need to do when you sign a new customer?
• Does your system support cobranding?
• How do you support multiple locales, languages, and cultures?
• How do you support both a hosted and a shrink-wrap delivery model?
• Does your bidding system support real-time “push” of bid information? How so?

Software delivery experience:


• How many users access your hosted system per day?
• How many concurrent users do you see in a typical day (on hosted system)?
• How many concurrent users is the hosted system is designed to support?
• How many customers/users are using your system in their own data centers?

Technical operations and data center:


• Describe the network and systems architecture for your hosted product
o Network architecture
 Load balancing systems and firewalls
 Network monitoring systems
 Intrusion detection/prevention systems

1 Confidential 16-Oct-10
Assessment

 ISP connectivity
o Server/system architecture
 Server brands and models
 Operating systems (version, release, service pack/patch level)
o Database servers
 Hardware and OS (Company, model, version, patch/SP level)
 Database vendor, version, patch/service pack level
• Describe what you have done from a data center perspective to provide:
o Scalability
o High availability
o Detection and/or automated repair of outages
• Describe and specify your SOP for moving software from development to QA to production
• Describe and define your SOP for changes to data center configurations
• Do you offer your hosted customers Service Level Agreements (SLAs) ?
o If so, what are the key parameters of those agreements?
o How often have you failed to live up to these SLA’s?
o What have been the causes of any failures to live up to these agreements?
• Describe your SOP’s for handling customer outages
o For your hosted customers
o For customers with your software installed behind their firewalls
 (Tier 1, 2, and 3 support and issue resolution)
o Do you have a formal Rapid Response Plan for handling outages and issues?
 Can you describe it in detail?
 Is it documented?
 Can we see it?

Security:
• Describe your overall approach to and philosophy about security.
• How do you prevent and detect hacking exploits?
o Denials of service
o Network intrusion
o Data theft
o Social engineering
• Who has access to the servers and networking equipment in your data center?
o Physical access
o LAN access
o Remote access (e.g. VPN or dialup)
• What type of audit system do you have in place?
o For changes made to the operating environment
o For changes made by those with “Administrator” or “root” access?

Technology inventory:
• Identify all language(s), development tools and 3rd party components/tools used
• Identify server platforms supported
o OS’s supported (Company, versions, service pack levels)
o Server hardware requirements and supported platforms
o Network infrastructure requirements
o Databases
 Primary DB’s supported
 What you run on in the hosted environment
 What your development team develops on
• Client platforms supported
o Operating systems (Company, versions, service pack levels)

2 Confidential 16-Oct-10
Assessment

o Hardware platforms supported


o Browsers supported (Company, versions, service pack and encryption levels)
o What is required on the client side to run your software?
 (D)HTML support? How do you support multiple browsers?
 Java applets?
 ActiveX controls?
 Downloaded executable applications (Java or otherwise)?
o Do you maintain “connections” between the client and the server?
 If so, how do you do it?
 Do you use a third-party product for this?

2) Product
Initial Application Set-up
• How would a customer first set-up their site for use?
• What do they need to do?
• What is done for professional services?

Ongoing Application Administration


• What ongoing administrative activities can the customer perform?
• What ongoing administrative activities must you perform?

Loading of Data into Application


• How does custom get usable spend and sourcing data into the site?
• How do you (or customer) ensure the data is clean and normalized?
• How is data on the site maintained over time so that is it clean and accurate?

Item (Catalog) Management


• How are the technical / specification details of the items that will be sourced loaded into the site? Who has
ability to add / change information?
• How do these become part of a sourcing project?
• Does item management support multiple level Bill Of Material items?

Supplier Management
• How is supplier information (e.g. contact, performance, capabilities) added in the application?
• Who has ability to add/ change information? Can supplier directly?
• How do suppliers become part of the sourcing project?
• How is this data kept clean and normalized over time?

Sourcing Project(s)
• Can application support Request for Information capability? If so, how does this work?
• Can application support Request for Quote / Proposal capability? If so, how does this work?
• Can application support multi-line real time auctions? If so, how does this operate?
• Can a Request for Quote / Proposal be automatically transferred to the auction without reloading
information?
• Can a sourcing project incorporating either an RFI, RFP/Q, auction be set-up including multiple users with
different roles and permissions?
• Is sourcing project bidding supported by an decision making optimization function? How does it work?
• Does the RFx support multi-line, multi-variable, multi-version and multi-round capabilities? If so can you
import n-level BOM?
• Does the RFx support both qualitative and quantitative parameters?

Savings Implementation
• Do you track the progress of identified savings to purchase order impact savings?
• Do you provide tools that help speed this process along?
3 Confidential 16-Oct-10
Assessment

• Does this tie to a contract management system?

Contract Management
• Does application support the creation, storage, and viewing of contracts? How does this work?
• Does application link supplier compliance to contract terms? How does this work?

Knowledge Management
• Does the application have a decision support tool to guide buyers to the right sourcing process or strategy?
How does this work?
• Does the application have knowledge management functionality? What do you include in “Knowledge
Management”? How does this work?

Training / Help
• How do users learn how to use the application?
• What online help is available? What is available to suppliers?

Globalization Capabilities
• What languages does the application currently support?
• Can you handle multiple time zones; number formats, and currencies, data formats in application? If so,
how?

Reporting & Search


• How does your reporting function work?
• Do you have standard reports, ad-hoc reports, or both? How are these activated? Is it by user or by
professional services?
• What options does a user have to search for information? What methods are used?
• Can you search on user custom defined fields? If so, how? What information items?

Project Management
• How are projects structured and managed in application?
• Are projects tied to an underlying workflow? How does this work?

Workflow
• How does your workflow function? What do you mean by “workflow”?
• Who configures workflow? Can customers or is it solely professional services?

Documents & Templates


• How are documents loaded, stored, and utilized within application?
• Do you have “templates”? What does this term mean to you? What templates do you have? How are these
created, stored, and utilized within the application?

Collaboration and Personalization


• Do you support message boards? How work?
• Do you support online “chat”? How work?
• Do you support virtual real-time meetings? How work?
• Do you support instant alerts? How work?
• Can individual users alter how information is displayed throughout the site?

Permissions / Roles
• Does application support flexible definition of user groups? Permissions? Which permissions are
supported?
• Can permission structure match an organization’s “org” structure? How does this work? How many levels
deep? Wide?

4 Confidential 16-Oct-10
Assessment

Supporting Services
• What supporting services do you offer? (e.g. training, professional services)
• What kind of technical support do you offer? What are its hours of operation?
• How do you host (if hosted software available)? Where?

Supplier Functionality
• What does suppliers do specifically in the sourcing project application area?
• What application administrative privileges do they have?

Integration
• Are all the modules integrated?
• Can they be installed individually in conjunction with the platform?
• What ERP, PS, MRP systems does the platform integrate with? How does it integrate (XML docs, Connector
Development Kit)?

Product Management Process Review


• What is your overall Product Management Process?
• How and from whom do you capture requirements for your products? How frequently is this done? What do
you consider your top three sources of information?
• How do you capture ongoing customer satisfaction?
• Do you have a formal customer “win/loss” program in place?
• How do you prioritize your product requirements?
• Do you write formal business plans? When is this done?
• How do you decide if a new product will be funded?
• How do you decide which requirements will be put in a new release of an existing product?
• Do you prototype / mock-up ideas and share with potential users on a regular basis?
• Do you have a formal user-testing program in place?
• Do you write formal Marketing Requirements Documents? What do these contain? Who uses these
documents?
• Do you maintain formal user groups or focus groups?
• What is your product rollout process?
• How do you manage your rollout process? Who manages overall rollout?
• How do you train your channel(s) (e.g. sales, partners)
• What is product management’s roll during a product “Beta”?
• Who do you consider your competition? Do you have formal processes in place to capture intelligence on
them?
• Do you maintain a formal analyst relations program?
• Do you have a formal Product Marketing Function? How does this interact with Product Management?
• How is your corporate marketing handled (e.g. collateral, demos, advertising, trade-shows, etc?) How does
this function interact with Product Management?
• Who do you directly work with in Development? In Pre-sales? In Professional Services? In Sales?
• Who officially decides when a product is ready “to ship” or “go live”?

3) Finance
"First" tier of requests:
• Historical financial statements by quarter
• Historical headcount info by function
• Equity structure & financing history
• As much detail as possible on R&D:
o Costs
o Roles
o Consultants
• Summary of debt & lease arrangements

5 Confidential 16-Oct-10
Assessment

• F/S and headcount projections (at least 1 year)


• Pending and threatened lawsuits

“Second” tier of requests:


• Audit results, if any (external financial auditors, IRS, state authorities, etc.)
• Historical revenue by customer
• Customer pipeline
• Sales force productivity / quota
• # of consultants / summary of significant arrangements with non-employees

4) Sales
Current Customers
• Contracts with current customers
• Current deals under negotiation
• Sales plan for 2002
• Use of partners in selling or implementation
• Description of pricing and revenue model

Sales pipeline
• Current number of salespeople
• Quota for each sales person
• Sales strategy
• Sales training

5) Organization
• Current organization chart
• Biographies of key team members, history with company
• Significant employee arrangements
• Performance management process
• Company values and character traits, if identified

6 Confidential 16-Oct-10